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Summary 

Center TRACON Automation System 
(CTAS)/Flight Management System (FMS) 
integration on the flightdeck implies flight 
crews flying coupled in highly automated 
FMS modes [i.e. Vertical Navigation (VNAV) 
and Lateral Navigation (LNAV)] from top of 
descent to the final approach phase of flight. 
Pilots may also have to make FMS route edits 
and respond to datalink clearances in the 
Terminal Radar Approach Control (TRACON) 
airspace. This full mission simulator study 
addresses how the introduction of these FMS 
descent procedures affect crew activities, 
workload, and performance. It also assesses 
crew acceptance of these procedures. Results 
indicate that the number of crew activities and 
workload ratings are significantly reduced 
below current day levels when FMS 
procedures can be flown uninterrupted, but 
that activity numbers increase significantly 
above current day levels and workload ratings 
return to current day levels when FMS 
procedures are interrupted by common ATC 
interventions and CTAS routing advisories. 
Crew performance showed some problems 
with speed control during FMS procedures. 
Crew acceptance of the FMS procedures and 
route modification requirements was generally 
high; a minority of crews expressed concerns 
about use of VNAV in the TRACON airspace. 
Suggestions for future study are discussed. 

Background 

National Airspace System Problems 

Increasing congestion at airports 
“By the year 2011, aircraft operations at the 
top 100 airports are projected to increase 32 
percent beyond current day (1998) 
operations. This increase and projected further 
increases indicate that the busiest U.S. airports 
are getting busier, which will compound 
problems of congestion at these key airports 
unless airport and airspace capacity 


enhancements tire made. Assuring that the 
capacity of the National Airspace System 
(NAS) can accommodate the growing demand 
for aviation services is critical to the Nation’s 
economic future” (FAA Office of System 
Capacity, 1998). 

Current operations in and near the terminal 
airspace 

The current air traffic control system uses 
analog voice communications, and tactical 
vectoring for aircraft separation and traffic 
flow management in the arrival and approach 
phases of flight. Aircraft in general begin their 
descent in an Air Route Traffic Control Center 
(ARTCC or Center) on charted Standard 
Terminal Arrival Routes (STARs) that include 
both lateral and vertical constraints for the 
path of flight. The lateral constraints consist of 
a path defined by straight line segments 
between navigational database waypoints that 
terminate at or near a “comer post” or entry 
gate into Terminal Radar Approach Control 
(TRACON) airspace. This lateral path defined 
on the STAR is generally cleared by Center 
controllers and flown by pilots as specified on 
the STAR chart. The vertical constraints of a 
STAR are specified as speed and altitude 
restrictions associated with the navigational 
waypoints on the STAR. When an aircraft 
nears the termination point of the STAR, it is 
handed off to TRACON air traffic controllers. 
TRACON controllers guide the aircraft to an 
assigned runway by issuing tactical altitude, 
speed, and heading instructions. Unlike the 
STAR used in the center airspace, these 
TRACON routes are not charted. Although 
nominal paths are usually followed in the 
TRACON, the aircraft crew is generally only 
aware of their current instructions (altitude, 
speed, and heading). As traffic levels increase, 
both Center and TRACON controllers are 
forced to direct aircraft on radar vectors’ that 
deviate from STARs and the nominal 
preferred TRACON paths to runways. 

Current and forecasted problems 

This approach to air traffic management is 

becoming inefficient in meeting growing air 


traffic demands. The continued growth of air 
traffic at major airports has caused increases in 
air traffic delays and has put considerable 
stress on both existing air traffic control 
(ATC) systems and on the air traffic 
controllers themselves (Davis, et al., 1997). 

The projected increase in aircraft operations 
would produce undue strain on the current air 
traffic management system. A more efficient 
system is needed in order to accommodate the 
anticipated growth in operations. These 
concerns have motivated development of 
computerized controller aiding tools aimed at 
helping controllers cope with increased traffic 
demands. 

Problems from the user perspective 

The airborne Flight Management System 
(FMS) is a highly capable present day 
technology. An estimated 10,000 FMS 
equipped aircraft exist in the current 
worldwide fleet. Aircraft FMSs enable 
airplanes to compute and fly fuel efficient 
trajectories accurately. However, the FMS is 
seldom utilized within the TRACON airspace 
due to the current controlling method of 
tactical vectoring. This requires aircraft to 
remain in the air longer and fly less efficient 
descent profiles than is possible when the FMS 
is used. The result is significant airspace 
congestion and major airline costs due to fuel 
bum and added delays. 

Proposed Solutions 

Controller aiding 

To address these problems, NASA and the 
FAA have been developing the Center 
TRACON Automation System (CTAS). CTAS 
is set of ground-based decision support and 
aiding tools intended to assist controllers with 
aircraft sequencing, separation, flow control, 
scheduling, and trajectory prediction in and 
around the terminal area (see Erzberger et al., 
1993). CTAS is comprised of three primary 
tools. The first tool is the Traffic Management 
Advisor (TMA). The TMA computes and 
plans a landing time and sequence number for 
every aircraft approaching an airport while 


that aircraft is still in cruise flight. Next the 
Enroute Descent Advisor (E/DA), advises a 
fuel efficient descent which can include a 
particular routing, profile, or speed, in order to 
meet the arrival time at the TRACON meter fix 
computed by the TMA. As the aircraft crosses 
the meter fix and enters the TRACON airspace, 
TMA updates the sequencing and arrival time 
plan and assigns a runway for the aircraft. 
Next, while the aircraft is in the terminal 
airspace, the third CTAS tool, the Final 
Approach Spacing Tool (FAST), advises the 
controller in order to accurately space and 
sequence the aircraft in the TRACON airspace. 
FAST advisories may include speeds or 
particular routings for the aircraft to fly in 
order to meet the TMA computed threshold 
arrival time. 

The CTAS tools manage arrival traffic 
strategically to prevent airspace overload and 
reduce delays. By assigning each aircraft 
approaching the airport a “slot” defined by a 
scheduled arrival time at the runway threshold 
and various intermediate arrival times along 
the descent path, CTAS can optimize traffic 
flow to an airport. CTAS can then aid 
controllers in implementing this optimal traffic 
flow by providing strategic routing and speed 
advisories for aircraft to fly in order to meet 
their scheduled arrival times (see figure 1). 

CTAS/FMS integration 

The Enroute Descent Advisor tool has 
undergone field testing at Denver International 
Airport in recent years (see Green et al., 1996; 
Cashion et al., 1995; Palmer et al., 1997). The 
E/DA implementation utilized in these field 
tests involved issuance of top of descent point 
and cruise and descent speed advisories to 
aircraft. These advisories served to adjust 
arrival time at the TRACON metering fix while 
keeping aircraft on conflict free trajectories. A 
more advanced implementation of the E/DA to 
aid controllers with arrival time adjustments 
supports rerouting of the aircraft. This is 
accomplished with a Trial- 
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Figure 1. CTAS Tools: Traffic Management Advisor (TMA), Enroute Descent Advisor (E/DA), Final 
Approach Spacing Tool (FAST). 


Planning- Waypoint functionality that allows 
controllers to propose routings to the E/DA 
with a movable waypoint to adjust arrival times 
and avoid conflicts. This new routing, once 
verified by the E/DA and air traffic controller, 
can then be communicated to the aircraft by 
voice or by data link. The data link 
implementation would allow uplink of this 
route to Flight Management System (FMS) 
equipped aircraft. This E/DA functionality is 
proposed as a possible approach to integrating 
CTAS with the airborne FMS in the Center 
airspace. It is hypothesized that the uplink of 
these routes, to be loaded and then flown by 
the aircraft FMS, could improve aircraft 
flightpath accuracy and therefore increase 
arrival-time accuracy. 


The FAST tool can also be utilized to various 
degrees to aid controllers with scheduling and 
sequencing aircraft. The Passive FAST 
implementation, which has been under field 
study for the past few years at the Dallas Fort 
Worth Metroplex (see Davis et al., 1997), 
provides controllers with landing sequence and 
landing runway advisories for all aircraft 
entering the TRACON airspace. It is left to 
controllers’ discretion whether to heed these 
advisories. 

A more advanced functionality of FAST 
currently under research is known as TAP 
FAST. This FAST implementation not only 
provides the controller with a computed route 
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for each aircraft to fly in the TRACON 
airspace but provides the capability to uplink 
this route to Flight Management System 
(FMS) equipped aircraft. TAP FAST is 
proposed as a possible approach to integrating 
CTAS with the airborne FMS in the TRACON 
airspace. As with the E/DA functionality, it is 
hypothesized that the uplink of these routes to 
be flown by FMS aircraft could improve 
aircraft flightpath accuracy in the TRACON 
airspace to increase arrival time accuracy. This 
could allow for closer entrail spacing of 
aircraft and increase runway throughput at 
airports. This can also be down with the Active 
FAST system. 

In order to implement these E/DA and FAST 
route uplink functionalities as described, an 
air-ground data link capability is necessary. 
This would entail the implementation of many 
technologies not currently available to ATC 
ground stations, aircraft flight decks, and the 
National Airspace System (NAS) 
communications infrastructure. Hence, this use 
of these E/DA and FAST concepts is 
necessarily intended for more far term 
implementation. 

To permit CTAS/FMS integration in the near 
term, a concept has been developed that is 
intended to work with currently available 
technologies. This concept utilizes the E/DA 
Trial-Planning-Waypoint and TAP FAST 
functionalities minus their datalink capability. 
Thus, the controller is still provided with a 
route for each aircraft to fly but must 
communicate this route via the voice channel. 
Due to the complexity of information 
contained in even the shortest route segment, 
voice communication of all this information is 
not feasible. To make this concept workable, 
this study proposes configuring the E/DA and 
TAP FAST to compute nominal default paths 
from Standard Arrival Routes (STARS) to 
runway approaches utilizing only speed and 
path extension degrees of freedom. This 
permits the loading of these nominal routes in 
airborne FMS databases as well as charting 
these routes on cockpit Jeppesen charts. These 


charted routes are called FMS arrivals and 
transitions. The controller can clear an aircraft 
for the FMS arrival and transition and verbally 
specify any modifications to this route whether 
they be route modifications, speed change 
points, or path extension lengths (see 
Appendix A for a sample FMS arrival and 
transition chart). 

Coordination of automation tools such as 
E/DA, FAST, and the FMS may enable aircraft 
to use their FMS computed trajectories from 
top of descent to final approach. If an aircraft 
can remain on an FMS trajectory until the 
final approach, a number of sources of arrival 
time error are eliminated. This is because 
aircraft on FMS routes can fly more 
predictable and precise trajectories, and thus 
they are more likely to match trajectories and 
arrival times anticipated by controllers and 
predicted by the CTAS tools. Placing aircraft 
on FMS paths can also help reduce excess 
spacing buffers, and increase the efficiency of 
the CTAS delay distribution function 
(Couluris et al., 1997). This could allow for 
increased capacity, efficiency, and safety in the 
terminal airspace. Additionally, if controllers 
are able to clear aircraft for FMS routes in the 
Center and TRACON airspace, this would 
reduce the number of clearances required to 
guide aircraft through this airspace. This could 
result in a reduction in both pilot and 
controller communication workload. 

FMS Descent Procedures 

FMS descent procedures involve descents from 
cruise altitude to the final approach course 
intercept without radar vectors or altitude 
assignments with the aircraft coupled to the 
FMS in both the lateral and vertical navigation 
and guidance modes. The lateral and vertical 
route is specified on paper charts and is coded 
and selectable in the aircraft FMS electronic 
database. The route is loaded via the Control 
and Display Unit (CDU) and can be selected in 
the approach page database adjacent STAR 
and runway listings. As of 1999, STARs in the 
National Airspace System (NAS) generally 
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terminate at the TRACON entry “comer 
post” after which aircraft are typically issued 
vectors from ATC to the final approach 
course. FMS descent procedures vary in the 
extent of their routing but usually describe a 
path that includes the current day STAR 
routing and then extend into TRACON 
airspace to the point of localizer capture (final 
approach) at a specified runway or series of 
runways. FMS descent procedures are 
distinguished from STARs primarily by this 
charted transition routing in the TRACON 
airspace. Another distinguishing feature of 
FMS descent procedures is that when they are 
loaded into the flight computer via the CDU, 
all charted altitude and speed restrictions 
appear adjacent their corresponding waypoints 
on the CDU legs page. With STARs, charted 
crossing restrictions, whether specified as 
“expect” or “cross at” values, require 
manual input by the flight crew once the 
STAR has been loaded. When a STAR is in 
use, ATC is required to explicitly clear aircraft 
for all charted restrictions stated as “expect” 
on the STAR chart. FMS descent procedures 
do not include “expect” restrictions because 
all charted values, both speed and altitudes, are 
“hard” restrictions that must be flown once 
an aircraft is cleared for the FMS descent 
procedure. 

FMS descent procedures take advantage of the 
advanced Flight Management System in 
today’s aircraft. They are intended to optimize 
the lateral and vertical path flown by aircraft in 
order to reduce fuel consumption and provide 
a predictable descent path for aircrews and 
controllers thereby reducing the need for 
radar vectoring (Transport Canada, 1997). 
FMS descent procedures are designed to allow 
maximal utilization of the vertical and lateral 
navigation and guidance capabilities of 
automated aircraft. They provide the crew with 
a well defined predictable path for the aircraft 
to fly in the descent phase of flight. FMS 
descent procedures require flightcrews to use 
their FMS at low altitudes, in the high 
workload descent phase of flight. In today’s 
operations, crews are accustomed to a lesser 


degree of automation during this phase of 
flight that involves active speed, heading, and 
altitude control via direct autopilot inputs. 

FMS descent procedures are proposed as a 
primary integration step with the E/DA and 
FAST CTAS controller aiding tools. These 
have been termed CTAS/FMS procedures. This 
entails the FMS descent procedures described 
above with route modification and speed 
update capabilities added. This involves the 
flight crew receiving an initial clearance to fly 
the charted nominal FMS routing. While still 
in Center airspace, the crew may receive an 
updated routing from ATC, as advised by 
E/DA, in the form of a descent route 
modification. When entering the TRACON 
airspace, the crew is cleared for the remaining 
TRACON portion of the charted FMS routing. 
While flying this routing, the crew may receive 
an amended speed change point and/or a route 
modification (path extension), to adjust arrival 
time, spacing, or sequencing of aircraft as 
advised by FAST. The crew inputs this route 
modification into the FMS and proceeds to fly 
the updated routing. This procedure requires 
crews to fly coupled to the FMS in the Center 
and TRACON airspace and in addition to 
make edits to their FMS routes during this 
phase of flight. In today’s vectoring 
environment, speed amendments and path 
extension clearances are transparent to flight 
crews as they are essentially indistinguishable 
from nominal vectoring procedures and 
routes. 

Flying FMS and CTAS/FMS 
Descents 

Focus of Study 

The objective of this research is to determine 
how a shift from current day (as of 1999) 
descent procedures (STARs and tactical 
vectoring in the TRACON) to the proposed 
FMS and CTAS/FMS descent procedures will 
impact crew activities, workload, and 
performance. In addition, the study will elicit 
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pilot perspectives regarding the acceptability 
and usability of the proposed procedures. 

The introduction of FMS descent procedures 
and CTAS/FMS descent procedures may create 
substantial changes in the number and nature 
of the crew’s flightdeck activities. Current day 
descent procedures in the NAS only include 
routings for the initial descent segment within 
the Center airspace and require crews to fly the 
approach segment using discrete tactical 
clearances from ATC. This allows crews to fly 
the initial route segment of the descent in a 
highly automated mode with the autopilot 
coupled to the FMS, and requires less 
automated, tactical “Mode Control Panel 
(MCP) flying”, in the non-charted TRACON 
segment of the descent. FMS descent 
procedures require crews to fly coupled to the 
FMS, in a highly automated mode, for the 
entire descent until turning onto final 
approach. CTAS/FMS procedures impose the 
same requirements but also require crews to 
perform FMS route modifications during 
descent. These changes in flightdeck activities 
will be investigated through a full mission 
simulation of nominal (baseline) and complex 
descents for the current and proposed descent 
procedures. 

A change in the crew’s flight deck activities 
during the descent phase of flight may impact 
their workload. Descents are typically very 
busy periods for the flight crew, and concerns 
regarding high workload demands during this 
period are discussed in the literature (Wiener, 
1985). This study aims to determine if and 
how workload is affected by the proposed 
procedures. If significant shifts in workload 
levels are found, the goal will be to ascertain 
the nature of these shifts, whether the shifts 
maintain workload at acceptable levels, and to 
identify any potential problems or benefits 
that may emerge as a result of these shifts. 

Crew performance is another primary factor 
to consider with the introduction of new 
procedures. It is important to determine if 
crews can maintain acceptable performance 
when flying the proposed descent procedures. 


While workload is a metric indicating task 
difficulty, it is often dissociated from 
performance measures (Yeh & Wickens, 
1988). For example, in a situation where task 
demands (and thus workload) are already 
high with a given procedure, and the new 
procedure imposes even higher demands, 
performance may degrade but subjective 
workload measures may not increase. This is 
because nearly full resources are invested in 
the first procedure and thus no further 
resources can be invested into the more 
demanding procedure producing no 
subjective increase in task demand. 
Conversely, if task demands are already very 
low and increase yet remain low with 
introduction of the new task, performance will 
not change but subjective workload will 
increase (Yeh & Wickens, 1988). This 
possible dissociation makes it necessary to 
measure and compare both performance and 
workload for the new procedures. 

Study Approach 

The approach for this study was to develop 
candidate FMS and CTAS/FMS descent 
procedures for a single arrival, approach, and 
runway at the Dallas Fort-Worth Metroplex 
(DFW) that overlay the current day STAR and 
nominal TRACON vector routing. Then a set 
of descent scenarios was developed for current 
day, FMS, and CTAS/FMS operations. A 
nominal baseline descent scenario was 
generated for the current day (as of 1999) and 
IMS operations. A more complex descent 
scenario was generated for the current day, 
FMS, and CTAS/FMS operations. 

To support the newly developed CTAS/FMS 
procedures, various interface enhancements 
were designed and implemented. These are 
described below. A full-mission simulation was 
then conducted to track crews’ actual descent 
activities, and to gather workload, 
performance, and attitudinal measures from 
crews. 
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Development of FMS descent procedures 
The FMS descent procedures were developed 
by drafting an experimental Jeppesen chart 
(see Appendix A) and coding the charted 
routing in the FMS database of the test 
simulator. The arrival portion of the procedure 
overlays the routing of the current day Glen 
Rose 3 STAR at DFW (see Appendix A). 

Unlike the STAR, all the crossing restrictions 
are “hard” restrictions as depicted on the 
chart and are loaded into the FMS from the 
database. This means that the restrictions are 
an integrated element of the procedure and 
crews must comply with them when cleared for 
the procedure. The transition portion of the 
FMS procedure overlays the current nominal 
vector routing from the Glen Rose 3 STAR to 
runway 18R at DFW. Like the FMS arrival 
portion, this routing includes “hard” speed 
and altitude crossing restrictions up until the 
final approach course. Crews are required to 
fly the procedure in Vertical Navigation 
(VNAV) and Lateral Navigation (LNAV), the 
FMS guidance modes. 

Development of CTAS/FMS descent 
procedures 

The CTAS/FMS descent procedures (see 
Appendix A) are identical to the FMS descent 
procedures except for the addition of speed 
and route modification degrees of freedom as 
integrated elements of the procedure. Notes on 
the charted CTAS/FMS procedure advise the 
crew that they may receive a speed to fly 
and/or a route modification in the arrival 
segment of the descent. They are also advised 
that they may receive a speed amendment 
and/or a route modification (downwind 
extension) during the transition segment of the 
procedure. 

Interface enhancements 
To support the speed and route modification 
degrees of freedom required for the 
CTAS/FMS procedures, three cockpit interface 
enhancements were prototyped for the current 
study. The three enhancements represent a 


continuum from near term to far term 
technologies. 

The near term enhancement, termed Manual 
Path Extension (MPE), enables a route 
modification to the TRACON portion of the 
FMS procedure routing via CDU inputs made 
by the crew. The route modification involves 
an extension of the downwind and final 
approach distances of the FMS procedure. 

This is made possible through an experimental 
leg type termed a Fix-to-a-Variable-Distance 
leg. The length of this leg can be altered by 
the crew as many times as desired by down 
selecting its reference waypoint (ROSEL for 
current study) in the CDU scratch pad and 
entering the new desired distance. This is 
intended to support the TAP FAST tool which 
advises controllers with strategic base turn 
points for aircraft in the TRACON airspace. 

The second, medium term interface 
enhancement, termed CDU DL (Control and 
Display Unit Data Link), utilized data link . 
route modifications and communications via 
the CDU. This interface was inspired by the 
current FANS (Future Air Navigation System) 
data link implementation in use in the B747- 
400 aircraft. The CDU DL interface was used 
to support E/DA and FAST route 
modifications. Route modifications were 
uplinked to the aircraft and loadable directly 
into the FMS by crew CDU actions. This 
loadable route could be inserted into the 
current FMS route and thus implemented the 
specified route modification. 

The far term interface implementation, termed 
DDL (Distributed Data Link), also features an 
air-ground data link for route modification 
communications. Unlike the CDU DL 
interface, data link route messages were 
displayed directly in the upper center EICAS 
(Engine Indicating and Crew Alerting System) 
screen on the flightdeck, and responses to data 
link messages were made using buttons placed 
adjacent to the MCP on the flightdeck 
glareshield. Route modification messages 
automatically loaded into the FMS ‘mod 
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route’ upon arrival. Crews reviewed, and if 
they were acceptable, executed and accepted 
these route modifications. 

Each of these interfaces allows the actual 
computed CTAS routing to be input into the 
FMS. Because some of these route 
modifications occur in the busy TRACON 
airspace, key questions regarding each of these 
interfaces are: Will crews successfully perform 
all the necessary steps required to complete the 
route modification task for each interface? 
How much time will be required to complete 
the route modification tasks? How much total 
“heads-down” time will be required by each 
crew member during performance of the 
tasks? For each interface, what is the average 
length of time each crew member fixes his or 
her gaze, or dwells, in the “heads-down” 
position? 

Descent scenarios 

Two descent scenarios were developed for the 
study. The first scenario, termed baseline, 
consisted of ATC clearances to fly charted 
routings from top-of-descent to final approach 
with no deviations for vectors, temporary 
altitudes, or speed amendments. The second 
scenario, termed complex, contained deviations 
from the charted routings including vectors, 
intermediate altitude level-offs, and speed 
amendments. The complex scenario 
represented likely CTAS advisory clearances 
from ATC. These two descent scenarios were 
used in both current day and FMS procedure 


conditions to facilitate direct comparison 
between the current day (STAR) and FMS 
procedure contexts. The complex scenario was 
also used for the three CTAS/FMS procedure 
contexts (Manual Path Extension, CDU DL, 
and DDL.) to test use of these interfaces as a 
method of dynamically updating FMS routes 
(see figure 2). 

Theoretical Perspective of Crew Activity 
Analysis 

Prior to analysis of crew activities, a short 
background and theoretical discussion of pilot 
interaction with aircraft automation is 
necessary. This will provide a framework for 
discussion and analysis of crew activities. 

Control and management automation 
descriptions and definitions 

Automatic piloting devices have been in use in 
aircraft since the 1930’s. These early 
autopilots relieved pilots of the manual labor 
of hand-flying on long flights. They provided 
inner-loop control to the aircraft in response 
to direct pilot instructions but left the pilot to 
perform all navigation and other essential 
piloting tasks (Billings, 1996). By the 1980’s 
autopiloting advances built on microprocessor 
technology provided the capability to program 
complete routes into a flight computer, to 
couple this computer to an autopilot, and to 
hand over the navigation, guidance, and 
control tasks to the computer. 


Procedure 

Baseline Scenario 
(no route deviations) 

Complex Scenario 
(CTAS advisories) 

Current Day (as of 1999) 

X 

X 

FMS 

X 

X 

Manual Path Extension 


X 

CDU DL 


X 

DDL 


X 


Figure 2. Test matrix. 
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The crew acts to monitor the computer’ s 
“actions”, to update the programmed flight 
route, or make any necessary interventions. 
These two categories of automation have been 
termed control automation (e.g. Fadden, 

1990) and management automation (e.g. 
Billings, 1996) respectively (see Billings 
(1996) for a more complete discussion). 

In the aviation domain, control automation 
entails the use of an autopilot as a tactical 
controller of the aircraft in response to explicit 
speed, heading, and altitude command inputs 
from the pilot. Management automation 
involves strategic programming of a complete 
lateral and/or vertical flight plan into a flight 
management computer which can be coupled 
to the autopilot to autonomously carry out the 
aircraft navigation, guidance, and control 
necessary to follow the programmed route 
with limited pilot input. 

With an aircraft under control automation, the 
pilot makes discrete tactical inputs 
commanding immediate output (behavior) 
from the autopilot. The crew ensures that the 
autopilot is moving toward and then 
maintaining this goal (monitoring) until the 
time for further tactical input arises. For 
example, the pilot may select a new heading 
for the aircraft to fly by dialing the value into 
the autopilot (Mode Control Panel). The 
autopilot in turn commands the aircraft to turn 
immediately to this heading and maintain it 
until the pilot makes further tactical inputs. 
Thus, control automation is characterized by 
the pilot specifying a discrete goal state to the 
autopilot and the autopilot proceeding 
immediately to satisfy and maintain that goal. 

With an aircraft under management 
automation, the pilot makes primarily strategic 
inputs to a flight management computer 
(FMC). The pilot can then couple the FMC to 
the autopilot so that it carries out the 
programmed inputs. The pilot then must 
verify that the aircraft is behaving as 
programmed (monitoring). For example, the 
pilot may input a series of waypoints that 


define a route from one airport to another, 
couple the autopilot to the FMC via the lateral 
navigation mode (LNAV), and then monitor 
the aircraft behavior as it flies the programmed 
routing. Thus, management automation is 
characterized by the pilot specifying a series 
of goal states to the FMC in one programming 
cycle, and the FMC commanding the autopilot 
to carry out these goals without further pilot 
input. Unlike control automation, in 
management automation the aircraft does not 
necessarily proceed immediately with 
programmed goals. The aircraft can only 
pursue one goal (in each control axis) at a 
time and thus a latency often exists between 
goal specification and goal execution (aircraft 
behavior). 

For ease of discussion the phrase “tactical 
input” or “tactical activity” will be used 
interchangeably with the phrase “use of 
control automation”, and the phrase 
“strategic input” or “strategic activity” will 
be used interchangeably with the phrase “use 
of management automation”. 

Expected crew activity changes 
Based on activity models developed with 
subject matter experts prior to the simulation 
study, it is anticipated that compared to current 
day descent procedures, crews flying FMS 
descent procedures during the baseline 
scenario will show [hypothesis (1)] a marked 
reduction in the number of observable crew 
activities (physical inputs into aircraft 
automation) that results from [hypothesis (2)] 
performance of significantly fewer tactical 
activities (use of control automation), and 
therefore, performance of primarily strategic 
activities (use of management automation). 
However, when flying FMS descent procedures 
during the complex scenario, it is anticipated 
that [hypothesis (3)] the number of crew 
activities will be roughly equal to that of 
current day descent procedures. 

The anticipated changes in crew activities with 
the FMS procedure baseline scenario will alter 
crew flight deck monitoring behavior by 
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requiring intermittent and continuous 
monitoring of multiple latent aircraft 
behaviors typical of management automation 
use rather than immediate monitoring of 
immediate aircraft behaviors as is seen with use 
of control automation. Also, with the FMS 
procedures, crews will be entrusting 
automation guidance to comply with 
programmed altitude restrictions, rather than 
meeting charted restrictions tactically, and 
crews are expected to utilize the LNAV and 
VNAV automated flight modes in the 
TRACON airspace. These modes are not 
available with current day (as of 1999) STAR 
procedures. This study investigates the 
workload and performance impact of these 
procedural changes in the number and 
character of flight deck activities, and the 
acceptance of these changes by crews. 

The CTAS/FMS descent procedures are 
adjuncts to the FMS descent procedure 
designed to better accommodate the dynamic 
advisories generated by the E/DA and FAST 
controller aiding tools. Aside from the 
assigned descent speed in the CTAS/FMS 
procedures, the complex scenario events are 
identical between the FMS and CTAS/FMS 
procedures. The primary difference is that the 
CTAS/FMS procedures permit crews to modify 
an existing FMS routing in a strategic fashion 
thereby retaining accurate FMS guidance of 
the flight. This is made possible in this study 
by the three interface enhancements 
previously discussed. 

The primary differences among the three 
CTAS/FMS procedures are the manner in 
which FMS route modifications are made. The 
interface enhancements provided in the 
CTAS/FMS procedures explore alternative 
methods for responding to the ATC 
prompting events. In this study, the 
acceptability and performance by flight crews 
of these route modifications and their 
interfaces is explored. A description of each 
CTAS/FMS procedure interface is provided 
below. 


The first CTAS/FMS interface, Manual Path 
Extension (MPE), involves manipulation of the 
existing FMS transition procedure base turn 
point via the CDU LEGS page. There are six 
tasks necessary to complete the activity. The 
crew member must (1) acknowledge and 
readback the ATC cleared base turn distance, 
(2) line select the reference waypoint ROSEL 
into the CDU scratchpad, (3) enter a backslash 
(/) and the new base turn point value, (4) 
reselect the waypoint into the flightplan, (5) 
review the modification on the CDU and 
Navigation display, and (6) execute the 
modification by pressing the execute button 
on the CDU. Observe that this activity requires 
the crew to make entries of numerical data by 
hand in a specified entry format (i.e. /X.X) 
making it potentially error prone and subject 
to forgetting (Billings, 1996). In addition, this 
is a CDU task that requires the non-flying crew 
member to go “heads-down” in the busy 
TRACON airspace. The MPE interface does 
not support E/DA route modifications issued 
in the Center airspace. 

The second CTAS/FMS interface, CDU DL, 
involves loading, executing, and accepting, via 
the CDU, an ATC data link message that 
contains an E/DA rerouting of the FMS Arrival 
or a FAST base tum point extension routing. 
This activity contains seven tasks. Upon receipt 
of the data link message (indicated by a chime, 
and an EICAS and CDU message), the non- 
flying crew member must ( 1 ) press the ATC 
key on the CDU, (2) read the message text on 
the CDU page aloud to the other crew 
member, (3) receive confirmation from the 
other crew member, (4) load the new route by 
pressing a LOAD prompt, (5) the flying crew 
member reviews and verifies the loaded 
message on the LEGS page (this is necessary 
because the non-flying crew member is on the 
ATC page) and Navigation display, (6) the 
non-flying crew member accepts the message 
by pressing the ACCEPT prompt, and (7) the 
flying crew member executes the new route. 

The CDU DL activity requires both crew 
members to go “heads-down” in the 
TRACON airspace. 
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The third CTAS/FMS interface, DDL, involves 
executing and accepting an ATC data link 
message through use of a “heads-forward” 
interface. This activity contains five tasks. 

Upon receipt of the data link message 
(indicated by a chime, and an EICAS and 
CDU message) the non-flying crew member 
must (1) read the message text on the EICAS 
display aloud to the other crew member, (2) 
receive confirmation from the other crew 
member, (3) review and verify the 
automatically loaded message on the LEGS 
page and Navigation display, (4) execute the 
new route by pressing the execute button on 
the CDU, and (5) accept the message by 
pressing the accept button on the glareshield. 
Unlike all the tasks in the previous two 
activities, only task (4) requires the non-flying 
crew member to go “heads-down” in this 
activity. Furthermore, when the message 
arrives, the text is immediately displayed on 
the EICAS screen, and the route modification 
is automatically loaded. This eliminates the 
access and loading steps required with the 
CDU DL activity. Both the CDU DL and DDL 
activities eliminate the requirement to 
manually enter new route data as in the MPE 
condition. 

It is important to note a distinction between the 
MPE and data link conditions. With the MPE 
path extension, crews manually modify only 
the base turn point. In the data link conditions, 
the FAST uplink may contain other changes to 
the route in addition to the base turn point 
extension such as a modified speed schedule 
or new crossing restriction parameters. 

Activity models generated with subject matter 
experts reveal a modest reduction in the 
number of required crew activities with the 
introduction of the data link interfaces only. 
The reduced number of activities for the CDU 
DL and the DDL procedures result from the 
uplink of CTAS route segments that provide 
modifications to the FMS descent procedure 
routing. This permits crews to remain coupled 
to their FMS routing, and thus to continue 


strategic flying, through most of the descent. 
The number of crew activities for the MPE 
procedure is nearly identical to the complex 
FMS procedure scenario for the initial descent 
segment. Differences in activities arise in the 
TRACON airspace. The MPE interface 
supports strategic modification of the base 
turn point distance. This prevents the need to 
revert to control automation as in the FMS 
procedure when the base tum point is 
extended by ATC. 

Relationship to previous research 
Since the advent of automation technology in 
commercial aircraft, a steady stream of 
research has emerged that addresses the impact 
of automation on flight crews (Wiener & 

Curry, 1980; Wiener, 1985; Curry, 1985; 
Wiener, 1989; Tsang & Vidulich, 1989; 
Billings, 1996). Two main assertions appear 
repeatedly in this body of literature. First is the 
assertion that the human is poorly suited as a 
system monitor. And second is the claim that 
increased automation on the flight deck may 
not reduce crew workload but will merely shift 
its nature from physical operations (e.g. 
button presses, etc.) to more mental operations 
(e.g. monitoring, etc.). Billings (1996) 
discusses issues of monitoring and workload 
of management automation specifically. He 
remarks that use of management automation 
means management by exception. This means 
the automation is given the authority to carry 
out all programmed tasks and performs them 
unless the crew takes exception. He comments 
that this may lighten physical workload, but 
may increase the cognitive burden on the crew 
as they are required to anticipate and confirm 
correct automation behavior. Also, Billings 
warns that use of this type of automation may 
decrease the crew’s involvement with the 
flying task and the mission as a whole. He 
remarks that even the most motivated and 
capable pilot can fatigue and lose vigilance 
when activity demands are so dramatically 
reduced. While the claims in this literature 
hold theoretical and intuitive merit, empirical 
study is needed to support or oppose these 
assertions. This study is intended to provide 
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such empirical data. The workload and 
performance measures to be gathered are 
intended as an initial examination into the 
effects on pilots of increased automation in the 
descent phase of flight. The assertions that 
pilots are poorly suited as system monitors and 
that greater use of management automation 
may decrease pilot involvement in the flying 
task are not directly addressed in this study. 
However, performance results may be able to 
provide hints as to possible problems brought 
on by increased use of management 
automation during the descent phase of flight. 

Method 

Participants 

Twelve flight crews who fly for commercial air 
carriers were recruited to participate in the 
study. Participants were recruited by the 
Raytheon Corporation and were paid for their 
time and compensated for travel and 
accommodation expenses. Crew participation 
requirements included: a type rating on the 
Boeing 757, 767, 737-500, or 777 aircraft and 
no prior data link experience. 

Equipment 

Crews flew in the Advanced Concepts Flight 
Simulator (ACFS) at NASA Ames Research 
Center (see Blake, 1996). This simulator is a 
full-mission type with a “generic” glass 
cockpit layout. It resembles the B757 aircraft 
in its avionics layout and functionality. 
Additions to the cockpit avionics are four 
interface enhancements: a vertical situation 
display (see Prevot, et. al. 1998); a variable- 
fix-to-a-distance leg type in the FMS database; 
a FANS data link functionality; and an 
advanced “heads-forward” data link 
interface. 

Design and Procedure 

After reading and signing a consent form (see 
Appendix B), each crew flew seven descents 
(due to time constraints, four current day and 
six FMS descents were not flown) from the 
descent procedures and scenarios previously 


discussed (see figure 2 and Appendices A and 
C). Twelve unique random orders were 
arranged for the seven descents with the 
condition that no descent appear first more 
than twice. Crew members alternated as pilot- 
flying (PF) and pilot-not-flying (PNF) for 
each descent. Two pseudo controllers acted as 
confederates and provided ATC clearances to 
the crews from an ATC laboratory at the 
simulator facility. A pseudo pilot acted as a 
confederate and interacted with the pseudo 
controllers to simulate background chatter 
during the descent runs. 

Crew training 

Crews underwent a training session prior to 
beginning each descent procedure. The 
training session included presentation and 
explanation of procedure charts and activities 
on overhead slides and crew performance of 
activities in the ACFS with the simulator in 
“flight freeze”. The training criterion was that 
the crews successfully performed procedure 
interface activities (e.g. MPE, CDU DL, and 
DDL) once before the data collection runs 
were flown. In addition to the procedure- 
specific training sessions, crews received a 
background briefing on the CTAS/FMS 
integration concept. Also, six of the twelve 
crews received training that allowed them to 
use the VSD during the study. 

Dependent Measures 

Crew activity analysis 

The number and type of crew activities 
performed during descents within each 
procedure condition and scenario type were 
gathered using the Crew Activity Tracking 
System (CATS) (Callantine, et. al., 1999). 
Comparisons were made between the baseline 
and complex scenarios of the current day and 
FMS procedures and complex scenario of the 
FMS and CTAS/FMS procedure scenarios. 

Crew workload measure 

Following each descent crews were asked to fill 
out the NASA-TLX subjective workload rating 
sheet (see Appendix D). Crews were briefed on 
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use of this instrument prior to flying the 
descents. Subjective crew workload is a 
dependent measure in the study. 

Workload definition and measure description 

Crew workload is a measure used in the 
aviation domain to assess the level of difficulty 
of required cockpit tasks and procedures. The 
difficulty of a task can be inferred from the 
interaction between an operator and the 
assigned task. Workload refers to the “cost” 
imposed on the operator as the task is 
performed (Gopher & Donchin, 1986). This 
implies that the operator has a limited 
information processing capacity with a given 
level of available resources (Kahneman, 1973) 
and that a given task has variable demands on 
those resources. Hence, workload can be 
thought of as the relationship between 
resource supply and task demand (Wickens, 
1992). The more operator resources are 
imposed on by a task, the less the resource 
supply will be, resulting in higher workload. 

Subjective workload measurement is aimed at 
capturing task demand and amount of 
operator resources expended (operator output) 
through operator self-report. Hart and 
Staveland (1988) developed the NASA Task 
Load Index (NASA-TLX), a multi- 
dimensional subjective workload measure that 
assesses perceived task demands across three 
dimensions and perceived operator outputs 
across three dimensions. The task demand 
dimensions are Mental Demand, Physical 
Demand, and Temporal Demand. The 
operator output dimensions are Performance, 
Effort, and Frustration. Descriptions of these 
workload dimensions are found in Appendix 
D. 

Crew performance criteria 
Crew performance was evaluated on each 
descent based on crew compliance with 
procedural and ATC requirements. All 
requirements were enumerated during crew 
training sessions. A list of the specific 
performance criteria for each procedure can 


be found in Appendix E. This is a dependent 
measure in the study. 

Video and digital data 

To aid with crew performance data collection, 
crews were videotaped during performance of 
the descent procedures using an overall 
cockpit view and a “face” view tracking crew 
gazing behavior. The “face” view was used to 
track heads-down and dwell times during route 
modification activities. Additional cameras 
were placed on the MCP, CDUs, Primary Flight 
Display (PFD), and Navigation Display to 
collect crew activity data. Digital flight data 
was also recorded. 

Crew questionnaire 

Crew acceptance of the proposed procedures 
was evaluated with a post-experiment 
questionnaire and debriefing session. After 
flying the seven descent scenarios, crews were 
presented with the post-experiment 
questionnaire. Following completion of the 
questionnaire crews were interviewed and 
debriefed and allowed to ask any questions 
regarding the study. Results of the 
questionnaire are another dependent measure 
in the study. The crew questionnaire is shown 
in Appendix F. 

Results and Discussion 

In order to address the hypotheses described 
earlier, this section begins with analyses that 
make direct comparisons between the current 
day and FMS procedures in both the baseline 
and complex scenario conditions. The first 
analysis focuses on the changes in the number 
and type of crew activities. Activity changes 
between current day and FMS procedures in 
both the baseline and complex scenarios will 
be analyzed. This initial analysis will 
investigate the hypothesized activity changes 
previously mentioned. These are: (1) The 
expected reduction in the number of 
observable crew activities (physical inputs into 
aircraft automation) when crews fly FMS 
descent procedures during the baseline 
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scenario as compared to current day descent 
procedures. (2) This reduction is expected to 
result from significantly fewer tactical 
activities, and therefore, use of primarily 
strategic activities. (3) During the complex 
scenario, the number of crew activities in the 
FMS procedure will be roughly equal to that 
of current day descent procedures. Following 
the activity analysis, subjective workload 
ratings will be analyzed with the same 
comparisons used in the activity analysis. Next, 
two categories of crew performance will be 
assessed. The first category will address crew 
compliance with procedural elements such as 
automation use requirements and speed and 
altitude crossing restrictions. The second 
category will address crew performance during 
the CTAS/FMS procedure (complex) scenarios 
using the interface enhancements to perform 
TRACON route modifications during descent. 
Finally, selected elements from the post 
experiment questionnaire will be discussed. 

Crew Activities: Current Day vs. FMS 
for Baseline and Complex Scenarios 

Planned comparisons 

In order to determine the shift in the number 
and type of crew activities (automation inputs) 
with the introduction of FMS procedures, 
analyses were conducted using the Crew 
Activity Tracking System (Callantine, 1997). 

Of particular interest was the shift in activities 
with the introduction of FMS procedures as 
compared to current day procedures in both 
the baseline and complex descent scenarios 
used in the study. 

Number of activities 

To determine the differences in the total 
number of crew activities between the FMS 


and current day procedures and the baseline 
and current day scenarios, a 2 (Procedures) X 
2 (scenarios) analysis of variance (ANOVA) 
was conducted and revealed no significant 
main effect for procedure JF (1,11) = 1 . 7 , p > 
.05, but a significant main effect for scenario F 
(1,11) = 217.3, p.< .05, such that crews 
performed more activities overall during the 
complex scenario, M = 37 (S.E. = 1.1), than 
during the baseline scenario, M = 19 (S.E. = 

1 .3). The Procedure x Scenario interaction was 
statistically significant, F (1,11) = 39.7, p < 

.05, such that the pattern of activity count 
differences between the FMS and current day 
procedures depended on which scenario crews 
were flying. To further investigate this 
interaction and test the stated hypotheses, 
simple comparisons were performed between 
the procedures within each scenario. A one- 
way ANOVA between the FMS and current 
day procedures in the baseline scenario 
revealed, as expected [hypothesis (1)], that 
based on activity model predictions, crews 
performed significantly fewer activities on 
average when flying the baseline FMS 
procedure scenario, M = 14 (S.E. = 1.2) as 
compared to the baseline current day 
procedure scenario, M = 24 (S.E. = 1.4), F 
(1,11) = 42.6, p< .05 (see figure 3). However 
contrary to activity model predictions 
[hypothesis (3)] stating that activity counts 
would be roughly equal between current day 
and FMS procedures in the complex scenario, 
a one-way ANOVA in the complex scenario 
showed that crews performed significantly 
more activities during the FMS procedure, M 
= 41 (S.E. = 1.8), than during the current day 
procedure, M = 34 (S.E. = 1.6), F (1,11) = 14, 
p < .05. 
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This interaction reveals that when flying FMS 
descent procedures crews tend to make fewer 
automation inputs than in current day flying 
[hypothesis (1)] when the procedure can be 
flown uninterrupted. If interruptions to the 
FMS procedure are made, such as off-route 
vectors, intermediate altitude level-offs, and 
speed amendments, then crews tend to make 
more automation inputs when flying FMS 
procedures than with current day procedures. 
This is due to the necessity to shift back and 
forth between strategic and tactical automation 
modes when flying the FMS procedure. This 
finding is consistent with debriefing comments 
made by pilots who stated that FMS 
procedures are most desirable and require less 
automation interaction when ATC does not 
intervene with the charted routing. The 
changes in the number of specific activity 
types and the effects on workload of FMS 


procedures in both the baseline and complex 
scenarios are discussed below. 

Activity types 

Activity types used for this analysis were the 
strategic (management automation) and 
tactical (control automation) categories 
described above. Strategic inputs imply use of 
higher levels of automation that enable the 
aircraft to carryout future behaviors 
autonomously. For example, using the LNAV 
mode for lateral navigation. Tactical activities 
imply use of lower levels of automation with 
no autonomy granted to the automation other 
than to maintain the current tactical target. For 
example, using heading select mode for lateral 
navigation. 

To investigate differences in tactical activity 
counts between the FMS and current day 
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Baseline Scenario Complex Scenario 


Figure 3. Mean number of crew activities (+/- SE) in Current Day and FMS procedures 
for baseline and complex scenarios. 
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procedures during the baseline and complex 
scenarios and to test hypothesis (2), a 2 
(Procedures) X 2 (scenarios) analysis of 
variance (ANOVA) was conducted and 
revealed a significant main effect for 
procedure, F (1,11) = 22.4, p < .05, with crews 
performing significantly more tactical 
activities during the current day procedure, M 
= 16 ( S.E. = 0.9), over the FMS procedure, M 
= 13 ( S.E. = 1.5), a significant main effect for 
scenario, F (1,11) = 143.6, p < .05, such that 
crews performed significantly more tactical 
activities during the complex scenario, M = 20 
(S.E. = 0.5) over the baseline scenario, M = 10 
(S.E. = 0.9), and a significant interaction, F 
(1,11) = 18.0, p< .05, such that the pattern in 
the number of tactical activities performed 
between the two procedures depended on 
which scenario was being flown (see figure 4). 
To investigate this interaction further one-way 
ANOVAs were conducted between the two 
procedures within each scenario condition. As 
expected in the baseline scenario [hypothesis 
(2)], a one-way ANOVA showed that crews 
made significantly fewer tactical inputs in the 
FMS procedure, M = 6 (S.E. = 0.8), as 
compared to the current day procedure, M 
=13 (S^= 0.8), F (1,11) = 31.8, p< .05, but 
in the complex scenario, there were no 
significant differences, p > .05, in the number 
of tactical activities between the current day, M 
= 20 (S. E. = 0.7), and FMS, M = 20 (SJL = 
0.7), procedures. The drop in tactical activity 
count for the FMS procedure in the baseline 
scenario is congruent with the overall activity 
count pattern for the baseline scenario, but the 
equal procedure activity count means in the 
complex scenario does not reflect the rise in 
the number of FMS activities in this scenario 
in the overall activity count. A look at the 
strategic activity counts should clear up this 
discrepancy and complete the picture. 

To investigate the differences in the strategic 
activity counts between the FMS and current 


day procedures during the baseline and 
complex scenarios, a 2 (Procedures) X 2 
(scenarios) analysis of variance (ANOVA) was 
conducted and revealed significant main 
effects for procedure, F (1,11) = 8.5, p < .05, 
with crews performing significantly more 
strategic activities during the FMS procedure, 
M = 14 (S.E. = 1.5), over the current day 
procedures, M = 13 (S.E. = 0.5), a significant 
main effect for scenario, F (1,11) = 142.3, p < 
.05, with crews performing significantly more 
strategic activities during the complex 
scenario, M = 17 (S.E. = 1.0), than during the 
baseline scenario, M = 10 (S.E. = 0.6), and a 
significant interaction, F (1,11) = 46.0, p < .05, 
such that the number of strategic activities 
performed between the two procedures 
depended on the type of scenario being flown 
(see figure 5). To investigate this interaction, a 
one-way ANOVA of the number of strategic 
inputs made during the baseline scenario was 
conducted which revealed that crews made 
significantly fewer strategic inputs with the 
FMS procedure, M = 8 (S.E. = 0.8), than in 
the current day procedure, M =1 1 (S.E. = 

0-7), F (1,11) = 24.7, p < .05. However, the 
trend reversed in the complex scenario, where 
a one-way ANOVA revealed that crews made 
significantly more strategic inputs during the 
FMS procedure, M = 21 (S.E. = 1.2), than in 
the current day procedure, M =14 (S.E. = 

0.5), F (1,11) =, p < .05. This rise in the 
number of strategic FMS activities in the 
complex scenario, not found in the tactical 
activity counts, accounts for the overall activity 
interaction previously discussed, (see figure 
3). 

These results highlight the expected large 
decline in tactical inputs during descent in the 
baseline scenario with the introduction of FMS 
procedures. There was also a drop in the 
number of strategic inputs in the baseline 
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Baseline Scenario Complex Scenario 


Figure 4. Mean number of strategic crew activities in Current Day and FMS procedures for 
baseline and complex scenarios. 
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Figure 5. Mean number of tactical crew activities in Current Day and FMS procedures for 
baseline and complex scenarios. 
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scenario with the introduction of FMS 
procedures. With this drop in the number of 
inputs comes an increase in the number of 
latent behaviors that are relegated to the 
automation. Both these decreases led to a 
reduction in perceived workload in the 
baseline FMS scenario, but they raise questions 
as to the implications of increased automation. 
These issues are discussed below. 

The results also indicate that when crews are 
forced to deviate from and rejoin FMS 
procedures to comply with ATC constraints as 
in the complex scenario, the number of tactical 
activities required rises dramatically (from a 
mean of six to a mean of twenty) but matches 
current day procedure levels (twenty) for the 
same complex scenario. This dramatic rise for 
the FMS procedure is also found in the 
strategic activity counts. During the FMS 
procedure crews performed an average of 
eight strategic activities in the baseline scenario 
and an average of twenty-one in the complex 
scenario. However, unlike the pattern found in 
the tactical activity analysis, strategic activity 
levels in current day scenario do not rise so 
dramatically between the baseline and 
complex scenarios (from 1 1 to 14). This 
finding confirms the notion that during the 
complex scenario, where there are multiple 
interruptions to the FMS procedure, crews are 
required to make frequent strategic inputs, 
such as re-engaging LNAV and VNAV modes, 
in order to resume the procedure. This 
illustrates that because of the strategic nature 
of FMS procedures, any deviations from the 
planned and programmed trajectory can lead 
to a precipitous rise in the required crew 
activities, both tactical and strategic. This 
brings into question of the utility of 
continuing FMS procedures when deviations 
are required. This issue and the effects on 
workload of these activity changes is discussed 
below. 

Subjective Workload: Current Day vs. 
FMS for Baseline and Complex Scenarios 

Overall workload ratings were in the low range 
with a grand mean of 4.8 (S.E. = .4) over a 


one to twenty point range. Crews reported that 
lack of out-the-window traffic, no TCAS 
display, no adverse weather, minimal radio 
chatter, no company communications 
requirements, and no interruptions from the 
passenger cabin during the simulation to be 
the major factors influencing the lower 
workload ratings. Crews also commented that 
the lack of these factors detracted from the 
realism of the simulation. 

To determine workload differences between 
the two procedures within the two scenarios, a 
2 (procedure) X 2 (scenario) ANOVA was 
conducted which revealed no main effect for 
procedure, F (1,23) = 1.1, p > .05, a main 
effect for scenario, F (1,23) = 5.0, p < .05, 
such that crews reported higher workload 
during the complex scenario, M = 5.1 (S.E. = 
0.3), than during the baseline scenario, M = 

4.4 ( S.E. = 0.3), and a significant interaction, 

F (1,23) = 6.0, p < .05, such that the pattern of 
workload differences between procedures 
depended on the which scenario was being 
flown (see figure 6). 

Planned comparisons 

To investigate the nature of the interaction, the 
following workload comparisons were 
conducted based on comparisons of particular 
procedure scenarios from the activity analyses 
above. 

Baseline current day procedure scenario vs. 
baseline FMS procedure scenario 

A one-way ANOVA revealed that pilots 
reported significantly lower mean workload 
scores in the baseline FMS procedure scenario, 
M = 3.9 (S.E. = 0.5), than in the baseline 
current day procedure scenario, M = 5.0 (S.E. 

= 0.5), F (1,23) = 6.0, p < .05 (see figure 6). 

Complex current day procedure scenario vs. 
complex FMS procedure scenario 

A one-way ANOVA showed that pilots 
reported a non-significant difference in mean 
workload scores in the complex FMS 
condition, M = 5.2 (S.E. = 0.5), than in the 


18 


complex current day condition, M = 4.9 (S.E. 
= 0.5). F (1,23) = .18, p > .05 (see figure 6). 

For the baseline scenario, the lower mean 
activity count corresponds to the lower 
reported workload in the FMS procedure as 
compared to the current day procedure. But 
the higher mean activity count for the FMS 
procedure complex scenario does not show a 
corresponding significant increase in workload 
over the complex current day procedure 
scenario. 

These results indicate that use of FMS 
procedures during the descent phase of flight 
leads to reduced flight crew subjective 
workload ratings when these procedures can 
be flown uninterrupted. These procedures 
include significantly fewer crew automation 
inputs than current day procedures, and more 
use of management automation. The use of 
management automation includes use of the 
VNAV and LNAV modes throughout the 
descent up until glideslope capture. This is 
evidence that higher levels of automation use 
can lead to lower crew subjective workload. 


When FMS procedures are interrupted by 
common ATC clearance amendments, forcing 
crews to switch between high and low levels of 
automation multiple times, the average number 
of crew activities nearly triples (from a mean 
of 14 to a mean of 41) as compared to the 
current day rise from 24 to 34 activities. 
However, workload does not rise significantly 
above current day levels. This indicates that 
the requirement to shift between automation 
levels during descent does not necessarily 
increase the workload burden on flightcrews. 
During the debriefing sessions, many crews 
remarked that they are accustomed to making 
transitions between higher and lower levels of 
automation in their current day environment. 

The conclusion that workload is not increased 
when crews are required to frequently switch 
between management and control automation 
is made with caution due to the artificially low 
workload levels experienced by crews overall 
during the study. These findings need further 
validation from studies with more realistic task 
loading on flightcrews during the descent 
phase of flight. It should also be noted that a 
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Figure 6. Mean subjective workload ratings (+/- SE) for Current Day and FMS procedures in the 
baseline and complex scenarios. 
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reduction in workload is not necessarily the 
goal of these procedures. The objective was to 
ensure that with the introduction of FMS 
procedures, workload remains at manageable 
levels. These results suggest that this is the 
case. 

The current results indicate that if FMS 
procedures are implemented, then greater use 
of management automation in the descent 
phase of flight will result. The drop in 
workload levels during the baseline FMS 
procedures as compared to baseline current 
day procedures raises questions about the 
possibility of introducing excessively low 
workload levels during FMS procedures. The 
use of management automation assigns most 
flying tasks to the autopilot after the flight 
crew loads the FMS procedure and sets up the 
automation to carry-out the procedure 
parameters. The flight crew then acts primarily 
as a system monitor. This calls on concerns 
raised by Billings (1996) that greater use of 
management automation may decrease crew 
involvement in the flying task. This study does 
not directly address this issue. Performance 
results discussed below address how well crews 
comply with FMS procedure requirements, but 
give no indication of changes in crew vigilance 
due to increased monitoring demands imposed 
by FMS procedures. This issue requires the 
use of specific techniques to test crew vigilance 
such as the inclusion of automation errors or 
failures during use of these procedures. 

Crew Performance: Compliance with 
Procedural Elements 

The first set of performance results refers to 
how well crews complied with the requirements 
and constraints of the procedures flown. This 
includes compliance with speed and altitude 
constraints prescribed by each procedure. Also 
included are the operational requirements in 
the FMS (and CTAS/FMS) procedures to use 
VNAV and LNAV during the descent. 

Speed and altitude compliance 

In two of the seventy-four descents (2.7%), 
crews were more than ten knots fast at the 


TRACON boundary crossing restriction 
waypoint, FEVER. In two of the seventy-four 
descents (2.7%), one crew was more than 300 
feet high at the FEVER restriction and another 
crew was more than 300 feet low. In four of 
the seventy -four descents (5.4%), crews were at 
least ten knots fast at the DELMO waypoint 
along the TRACON routing. In 30 of the 54 
FMS procedure descents (56%), crews were at 
least ten knots fast as they flew the downwind 
approach routing during the FMS procedure. 
In 9 of the 74 descents flown (12.2%) crews 
were at least ten knots slow at the assigned 
final approach fix crossing speed. 

The high number of crew overspeeds (56%) 
observed during the downwind approach 
routing of the FMS procedure may be due to 
multiple factors. First, the charted vertical 
routing for the downwind leg of the FMS 
procedure requires a relatively steep angle of 
descent in order to comply with altitude 
crossing restrictions along the route. Second, 
when the aircraft is in the VNAV vertical 
mode, the automation prioritizes tracking of 
the computed path and does not track the 
programmed speed. These two factors 
combine so that, without pilot actions, the 
aircraft tends to overspeed during this 
segment. Crews had the option to extend flaps 
and/or speed brakes to keep the aircraft at 
required speeds. They were slow to use these 
controls in the overspeed cases. This may 
indicate that pilots are willing to sacrifice 
precision in flying charted speeds in order to 
meet altitude restrictions, save fuel, or maintain 
passenger comfort. Also, this portion of the 
routing may not be considered as speed 
constrained by pilots, because speed 
constraints are only specified at waypoint 
crossings. It is important for the accuracy of 
the CTAS system and the ATC advisories it 
computes that crews comply as closely as 
possible with charted and cleared speeds at and 
between waypoints. Therefore, the high 
number of crew overspeeds while on the FMS 
procedure during this portion of the flight is a 
concern. It brings into question the utility of 
requiring use of VNAV to descend on the 
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FMS procedure. The FLCH and V/S modes are 
alternative ways of descending the aircraft that 
puts more priority on speed tracking and thus 
may serve the needs of the CTAS system better 
than the VNAV descent mode. 

Nine of seventy-four crews (12.2%) crossed 
the final approach fix more than ten knots 
slower than the charted and cleared speed of 
170 knots. This speed is set for the CTAS 
system to precisely control an aircraft’s arrival 
time at the runway threshold. During the post 
experiment debriefings crews that failed to 
comply with this speed restriction mentioned 
two reasons for doing so. First these crews 
mentioned that they habitually set their 
approach reference speed into the autopilot 
upon receipt of the approach clearance by 
ATC. They also mentioned the requirement to 
be ’’stabilized on the approach” by the final 
approach fix when flying in instrument 
conditions. This means that they have the 
aircraft fully configured and that airspeed is 
equal to the approach reference speed. Given 
this requirement, crews said that they felt 
uncomfortable crossing this point at the higher 
assigned speed and preferred to be slowed 
down to their reference speed by this point. 


VNAV and LNAV use 

When flying the FMS procedures, crews were 
required to use the VNAV and LNAV 
autopilot modes. During particular parts of the 
complex scenario, use of these modes was 
prevented by ATC clearances that diverted the 
aircraft from the charted FMS procedure. 
Crews were eventually cleared to resume the 
FMS procedure following the diversions which 
reinstated the VNAV and LNAV requirements. 
In 1 1 of the 54 (20.4%) FMS descents flown, 
crews used autopilot modes other than VNAV 
(i.e. FLCH or V/S) to descend after being 
diverted from and then re-cleared for the FMS 
procedure. All crews used the LNAV autopilot 
mode during all 54 of the FMS procedures. 
These results indicate some crew resistance to 
use of VNAV, or preference for the FLCH 
and/or V/S modes, during the descent phase of 


flight. Crew responses on the post-experiment 
questionnaire indicate some crew discomfort 
with the VNAV mode in the TRACON 
airspace. This issue if discussed at length 
below. 

All crews used the VNAV top of descent point 
when required during the FMS procedures. 

Crew Performance: CTAS/FMS 
Procedures Used During Complex 
Scenarios 

In this section, the analysis shifts and addresses 
crew performance using the interface 
enhancements during the complex scenarios. 
Recall that these procedures are termed 
CTAS/FMS procedures. 

TRACON route modifications 
Crew performance of the CTAS route 
modifications was evaluated for the TRACON 
route modifications only. This approach was 
chosen to facilitate comparison among the 
three interface concepts (MPE, CDU DL, and 
DDL) in which crews flew the complex 
scenarios and were able to respond to ATC 
route modifications using these technologies. 
Recall that MPE was not used for the Center 
route modification, and that it was the most 
progressive procedure proposal in the study. 
To evaluate crew performance of the CTAS 
TRACON route modification interface 
activities, a number of measures were 
collected. First, an evaluation of crew 
performance of the particular elements of each 
activity was performed. This entailed viewing 
the videotape data to record the elements of 
each activity that crews performed and 
comparing this with the expected performance 
based on how crews were briefed and trained 
for each interface activity. This analysis also 
included the recording of any entry errors by 
crews. Second, the total time to complete the 
route modification activity was recorded. 

Third, the total heads down time for each crew 
member was determined by viewing the 
“face” cameras from the videotape data. 
Heads down time is defined here as the total 
amount of time that a crew member spends 
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with his/her gaze on the CDU during the route 
modification activity. Fourth, the total heads 
down time for each observation was broken 
down into a collection of dwell times. Dwell 
time is defined here as the duration of a single 
fixation on the CDU during performance of 
the route modification activity. The sum of 
these dwell times equals the total heads down 
time. Note that dwell time was so large 
compared to travel time that travel time was 
ignored in this analysis. 

Performance of activity elements 

All crews in the MPE interface condition 
performed the activity without any errors or 
element omissions. In the CDU DL interface 
condition, four of twelve pilots not flying 
(PNFs) (33%) omitted the step of reading the 
clearance text aloud upon receiving the data 
link message. Five of twelve crews (42%) 
executed the route modification before 
reviewing the newly loaded route. No entry 
errors were made. In the DDL interface 
condition, three of twelve PNFs (25%) failed to 
read the message text aloud upon receipt of 
the data link message, and three of twelve 
crews (25%) failed to review the route 
modification before executing and accepting 
it. One of the twelve DDL crews (8%) failed to 
both read the message text aloud and to review 
the route modification before executing and 
accepting it. No entry errors were made in the 
DDL interface condition. 

The cases in which the PNFs failed to read the 
data link message text aloud and where crews 
failed to review the message may indicate task 
shedding due to crew occupation with other 
tasks. This possibility cannot be evaluated in 
this study because continuous crew task 
tracking data was not collected. However the 
low workload study environment that crews 
reported is not congruent with the notion that 
crews were saturated with other tasks. This 
issue of task omissions may be addressed in 
future studies by introducing errors or 
unforeseen elements in route clearance 
messages. It should be noted that these omitted 
tasks were checking and monitoring tasks and 


were not essential for successful completion of 
the route modifications activities. This means 
that the system can work properly when pilots 
do not perform these tasks. This is true with 
most procedural aspects of flying, such as use 
of checklists. It emphasizes the necessity to 
make checking and monitoring during data 
link operations a required standard operating 
procedure for crews. 

Total activity time 

The mean total activity times in seconds for 
the three route modification interfaces were, M 
= 24.1 (SjL = 2.3) for MPE, M = 26.7 (S^E. 

= 2.7) for CDU DL, and M = 20.3 (SjL = 

2.1) for DDL. Although DDL took the 
shortest amount of time on average, a one-way 
ANOVA revealed that mean total activity time 
differences among the three interfaces were 
not statistically significant, F (2,22) = 2.04, p > 
.05. 

These route modification activities resulted in 
activity times in the range of twenty to twenty- 
seven seconds. This raises the question as to 
what amount of time to complete this activity 
is operationally acceptable. To the author’s 
knowledge, no objective standard has been set. 
Crews frequently comment that they prefer to 
minimize time programming their FMS in the 
terminal area because it distracts from the vital 
tasks of instrument and out-the-window 
monitoring required in this airspace. All 
participants in the current study found these 
interfaces to be operationally acceptable for 
route modifications in the terminal area. The 
only crew concerns focused on the MPE 
interface which crews felt produced excessive 
heads-down time. This will be discussed below 
with the heads-down time and dwell time 
results. As mentioned before, crew operational 
acceptance of these interfaces and activities 
based on this study should be taken with 
caution. It is necessary to further validate the 
use of these interfaces and activities in flight 
conditions that more closely match those 
found at busy airports. This includes 
simulations with out-the-window traffic, TCAS 
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traffic, and likely distractions encountered 
during descent into the terminal area. 

Total heads down time 
Mean total heads down times for the PNFs 
(crew member who performed that majority of 
elements for each route modification) were, M 
= 15.3 (S.E. = 1.8) for MPE, M = 18.2 (S.E. 

= 1.8) for CDU DL, and M = 6.3 (S.E. = 1.7) 
for DDL (see figure 7). The DDL interface 
required less than half the heads down time 
than did the MPE interface and roughly one- 
third the heads down time than did the CDU 
DL interface. A one-way ANOVA of heads- 
down time revealed significant differences 
among the three interfaces, F (2,22) = 13.3, p 

< .05. Simple comparison ANOVAs revealed 
that the DDL interface heads-down time was 
significantly shorter than both the MPE (F 

(1,11) = 16.9 , p< .05) and CDU DL (F (1,11) 
= 21.6, p < .05) interfaces. There was no 
significant difference between the MPE and 
CDU DL heads down times, F (1 , 1 1 ) = 1 .4, p > 
.05 

Mean total heads down time for the Pilots 
flying (PFs) were,^M = 5.2 (S.E. = 1.2) for 
MPE, M = 9.7 (S.E. = 1.4) for CDU DL, and 
M = 4.1 (S.E. = 1.2) for DDL (see figure 8). 
The DDL required the least amount of heads 
down time for the PF, a little more than a 
second less than the MPE interface, but more 
than five-and-a-half seconds less than the CDU 
DL interface. A one-way ANOVA of heads- 
down time revealed significant differences 
among the three interfaces, F (2,22) = 9.27, p 

< .05. Simple comparison ANOVAs revealed 
that the CDU DL interface heads-down time 
was significantly longer than both the MPE (F 

(1,11) = 17.9, p < .05) and the DDL (F (1,11) 
= 17.7, p < .05) interfaces. There was no 
significant difference in heads-down time for 
the PF between the MPE and DDL interfaces, F 

(1,1 1) = .42 , p > .05. 

Dwell time 

PNFs had the shortest mean dwell times while 
using the DDL interface^ = 2.2 (S.E. = .4), 
followed by the CDU DL interface M = 8.3 


(S.E. = 2.4). The longest dwell times for PNFs 
occurred with the MPE interface, M = 10.7 
(S.E. = 2.1) (see figure 7). A one-way 
ANOVA of dwell time revealed significant 
differences among the three interfaces, F 
(2,22) = 7.30, p < .05. Simple comparison 
ANOVAs revealed that the DDL interface had 
a significantly shorter mean dwell time than 
both the MPE (F (1,11) = 15.1, p < .05) and 
CDU DL (F (1,11) = 5.9, p < .05) interfaces. 
There was no significant difference in mean 
dwell time for the PNF between the MPE and 
CDU DL interfaces, F (1,1 1) = 1.2, p > .05 

The mean dwell times for the PFs were less 
varied than for the PNFs: M = 2.8 (S.E. = 0.9) 
for MPE, M = 2.5 (S.E. = 0.3) for CDU DL, 
and 2.0 (S.E. = 0.5) for DDL (see figure 8). A 
one-way ANOVA of dwell time revealed non- 
significant differences among the three 
interfaces, F (2,22) = .39, p > .05. 

The results indicate that the DDL route 
modification interface required the least 
amount of heads-down time for both PNFs 
and PFs among the three interfaces. As with 
task time, the question remains as to what 
amount of heads-down time is operationally 
acceptable. While it is difficult to establish an 
objective standard for what is acceptable, pilot 
comments point to the common sense answer 
of “the less heads-down time in the TRACON, 
the better”. Perhaps a more operationally 
significant measure of time taken away from 
instrument and out-the-window scan is dwell 
time, because it represents the duration of a 
fixation on the CDU. If dwell times are short 
despite long cumulative heads-down times, this 
may indicate that pilots are time sharing, or 
have the opportunity to time share the route 
modification task with instrument and out-the- 
window scanning tasks. To validate this 
assumption, a higher resolution analysis of 
crew gaze and fixation behavior is required. In 
the current study, the PNFs average dwell time 
was considerably shorter with the DDL 
interface in addition to the considerably 
shorter total heads-down time found with this 
interface. These results indicate that the DDL 
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Figure 7. Mean total heads-down time and dwell time (+/- SE) for pilots not flying. 
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Figure 8. Mean total heads-down time and dwell time (+/- SE) for pilots flying. 


interface accomplished the objective of 
keeping the PNF more heads-forward during 
the route modification activity. For the PFs 
average dwell times for all interfaces were 
around two seconds indicating a tendency for 


the PF to include observation of the route 
modification activity into the flight deck scan. 
However, average total heads-down time for 
the PFs was significantly longer with the CDU 
DL interface than with the others indicating a 
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tendency for the PF to dwell more frequently 
on the CDU during the route modification 
activity. 

Post Experiment Questionnaire 

The following results cover only selected items 
from the post-experiment questionnaire. A full 
listing of all pilot responses to the yes or no 
and Likert scale questions is listed in 
Appendix G 

Acceptability of FMS procedure 
Crews were asked to respond to a series of 
questions regarding the acceptability of the 
Glenn Rose FMS Arrival and Delmo FMS 
Transition Procedure. The following are the 
results from selected items. 

In response to the question: “Was any portion 
of the Glenn Rose FMS Arrival and Delmo 
FMS Transition Procedure unclear or 
confusing?”, four of the twenty-four pilots 
answered yes. The stated reasons for this 
confusion were , (1) “speed assignments and 
alterations were confusing at first, then after 
re-reading the notes it became clear.” (This 
comment refers to note #2 on the chart that 
states speed and downpath routing 
amendments do not cancel the FMS 
procedure, which is contrary to current 
procedures), (2) “The ... procedure notes 
were too long. Chart should be read quickly 
and not have to be studied.”, and (3) “(it was 
unclear) at what point to abandon the LNAV 
transition and switch to localizer for 
approach.” 

In response to the question: “How acceptable 
or unacceptable was the requirement to Use 
LNAV during The Delmo FMS Transition?”, 
seventeen crew members responded that it was 
very acceptable, six responded that it was 
somewhat acceptable, and one responded that 
it was borderline. In response to the question: 
“How comfortable or uncomfortable were 
you using LNAV during the Delmo FMS 
Transition?”, nineteen pilots responded that 
they were very comfortable and five 
responded that they were somewhat 
comfortable (see figures 9 and 10). In 


response to the question: “How acceptable or 
unacceptable was the 

requirement to use VNAV during the Delmo 
FMS Transition?”, eleven pilots said this was 
very acceptable, six said it was somewhat 
acceptable, three pilots said it was borderline, 
three said it was somewhat unacceptable, and 
one pilot said it was very. In response to the 
airspace, approximately one-quarter of the 
twenty-four pilots felt that use of VNAV in the 
question: “How comfortable or 
uncomfortable were you using VNAV during 
the Delmo FMS Transition?”, eleven pilots 
responded that they were very comfortable, six 
said they were somewhat comfortable, two said 
they were borderline comfortable, three 
responded that they were somewhat 
uncomfortable, and two said that they very 
uncomfortable (see figures 11 and 12). 

These results reveal that while pilots had no 
objections to use of LNAV in the TRACON 
airspace, approximately one-quarter of the 
twenty-four pilots felt that use of VNAV in the 
TRACON was unacceptable and that they were 
uncomfortable with it in some way. Pilots 
confirmed these views during post experiment 
debriefings. All pilots mentioned that use of 
LNAV was fine within the TRACON. Those 
pilots who found the use of VNAV in the 
TRACON unacceptable or problematic stated a 
number of reasons listed here: (1) Use of 
VNAV below 10,000 feet lowers workload too 
much which may produce complacency and 
make the crew feel out of the loop; (2) Use of 
VNAV requires increased monitoring and 
constant asking of the question: “Is it going to 
do it?”; (3) Use of VNAV is made difficult 
when changes are made to the charted routing, 
forcing interaction with the CDU resulting in 
heads-down in the TRACON; (4) There are 
too many speed and altitude crossing 
restrictions requiring constant cross checking 
with charts, the CDU, and aircraft performance 
increasing workload and heads-down time; 

(5) Using FLCH or V/S makes it easier to get 
what you want out of the airplane; (6) (I) feel 
more assured MCP flying in the TRACON, (7) 
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Figure 9. Pilot questionnaire responses to the question: How acceptable or unacceptable was the 
requirement to use LNAV in the TRACON airspace? 
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Figure 10. Pilot questionnaire responses to the question: How comfortable or uncomfortable were 
you using LNAV in the TRACON airspace? 

VNAV “jerks” the a/c around and does not must match the actual winds to get good 

provide smooth transitions between crossing performance from VNAV, they usually do not 

restrictions; (8) the wind model in the FMS 
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match; and (9) anti-ice changes the descent 
profile and FMS performance. 

These complaints about use of VNAV in the 
TRACON airspace are varied. Comments (1) 
and (2) seem to address the baseline scenario 
condition in which the crew is cleared for the 
entire vertical FMS routing without deviation. 
This minority of crews reported feeling out of 
the loop and complacent when they delegated 
full vertical guidance to the automation which 
required vigilant monitoring to ensure the 
VNAV function meets all the charted 
restrictions. These comments echo Billings 
(1996) statements regarding use of 
management automation as increasing 
cognitive (monitoring) load and potentially 
decreasing crews involvement in the flying 
task. 

Comments (3) and (4) refer to increases in 
workload and heads-down time with VNAV 
usage when the route is complex or altered by 
ATC. This suggests the need to keep charted 
routes simple and quickly and easily 
decipherable, and to create effective route 
modification interfaces that minimize heads- 
down time and clearly and efficiently 
highlight changes to the FMS routing. 

Comments (5) and (6) refer to pilot 
preferences for the less automated vertical 
modes FLCH and V/S. Pilots report that these 
modes provide more precise speed control 
than VNAV and also provide the current speed 
target in the MCP speed window for quick 
heads-up reference and manipulation. 

Comments (7) through (9) refer to side-effects 
of VNAV use that degrade its performance 
and precision. 

In response to the question: “In General, how 
was your workload affected when flying the 
Glenn Rose FMS Arrival and Delmo FMS 
Transition as compared to your current day 
descent procedures”, four pilots responded 
that their workload was greatly decreased, nine 
pilots responded that their workload was 


somewhat decreased, five said that their 
workload was unaffected, and four responded 
that their workload was somewhat increased. In 
response to the question: “In general, how was 
you monitoring behavior affected when flying 
the Glenn Rose FMS Arrival and Delmo FMS 
Transition procedure as compared to your 
current day descent procedures?”, three pilots 
said that their monitoring was greatly 
increased, nine said that it was somewhat 
increased, eight said that their monitoring 
behavior was unaffected, and three said that 
their monitoring behavior was somewhat 
decreased (see figures 13 and 14). 

Pilot responses to these two questions indicate 
that the FMS procedures generally reduce 
workload while increasing monitoring 
demands. This dissociation between workload 
and monitoring reports indicates that pilots 
perceive an increase in monitoring demands to 
create a decrease in workload. This explains 
the complacency and out-of-the-loop 
experienced by crews during the FMS 
procedures. 

In response to the question: “How acceptable 
or unacceptable was the Glenn Rose FMS 
Arrival and Delmo FMS transition procedure 
as a whole?” , ten pilots said it was very 
acceptable, eleven said it was somewhat 
acceptable, two said it was borderline, and one 
said it was somewhat unacceptable (see figure 
15). 

The following are typical favorable comments 
made about the FMS procedure: “Procedure 
was easy and straight forward. Any confusion 
or difficulties had to do with the airspeed and 
altitude requirements which required a 
different cockpit scan”; “The FMS procedure 
made situational awareness much better. 
Planning ahead was much simpler. It was very 
easy to plan speed changes and flap/gear 
extension”. The following are typical 
cautionary or negative comments made about 
the FMS procedure: “...was not comfortable 
with the VNAV descent in the transition phase 
of flight”; “Using VNAV to fly the FMS 
transition was a bad idea”; “This procedure 
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Figure 11. Pilot questionnaire responses to the question: How acceptable or unacceptable was the 
requirement to use VNAV in the TRACON airspace? 
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Figure 12. Pilot questionnaire responses to the question: How comfortable or uncomfortable 
were you using VNAV in the TRACON airspace 

was not difficult or task overloaded. It doing what it is commanded. Setting the 

required a little more concentration in terms of altitude down to the final cleared altitude 

monitoring the automation to ensure it is 
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Figure 13. Pilot questionnaire responses to the question: In General, how was your workload 
affected when flying the Glenn Rose FMS Arrival and Delmo FMS Transition as compared to 
your current day descent procedures? 
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Figure 14. Pilot questionnaire responses to the question: In general, how was you monitoring 
behavior affected when flying the Glenn Rose FMS Arrival and Delmo FMS Transition 
procedure as compared to your current day descent procedures? 


could pose some problems with crews 
descending before “authorized” if they 
change to descent modes other than VNAV, or 
if descent mode changes automatically”; 
“Everything was okay. I still don’t completely 
trust VNAV in tight situations. LNAV is okay 
though”; “Too many speed restrictions and 
waypoints.” 


Acceptability of CTAS/FMS procedures 
The pilots were asked to respond to a series of 
questions to assess their acceptance and 
opinion of the CTAS/FMS procedures that 
they performed during the study. The 
following are the results from selected 
questionnaire items. 
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In response to the question: “How acceptable 
or unacceptable was the requirement to use 
data link for route modifications while in 
cruise near the top of descent?”, eighteen 
pilots said it was very acceptable and six said 
that it was somewhat acceptable. In response to 
the question: “How acceptable or 
unacceptable was the requirement to use data 
link for route modifications while in the 
TRACON?”, Twelve responded that it was 
very acceptable, nine said it was somewhat 
acceptable, two that it was borderline, and one 
pilot responded that it was somewhat 
unacceptable (see figure 16). This indicates 
that while all pilots felt comfortable with use of 
datalink before top of descent, three of twelve 
(25%) of pilots felt that use of datalink in the 
TRACON airspace was either borderline or 
somewhat unacceptable. Debriefing comments 
regarding this issue focused on the increased 
heads-down time required when processing 
datalink messages that pulled the PNF out of 
the instrument and traffic scanning tasks 
during a critical phase of flight. Pilots were 
asked to state the FMS route modification 
interface they most preferred. Fifteen said the 
DDL, seven the CDU DL interface, and one the 
MPE interface (see figure 16). The strong 
preference for the DDL is congruent with the 
shorter heads-down and dwell times required 
with this interface and pilot concerns about 
excessive heads-down time in the TRACON 
airspace. 

Conclusions and Future Work 

This study investigated the impact of FMS and 
CTAS/FMS descent procedures on crew 
activities, workload, and performance. It also 
assessed crew acceptance of these procedures. 

The findings can be summarized as follows: 

(1) The study demonstrated that use of FMS 
descent procedures can significantly reduce 


both crew activity load and workload below 
current day flying levels in situations where 
the charted procedure can be flown 
uninterrupted. However, activity loads 
significantly increase beyond current day 
levels and workload returns to current day 
levels when FMS descent procedures are 
interrupted by common ATC interventions 
and CTAS routing advisories. Further study is 
required to validate these conclusions and 
investigate issues that were not addressed in the 
current study. (2) Crew performance during 
use of FMS procedures was generally 
acceptable but suffered in a few key areas. 
More than half of the crews demonstrated 
poor speed control between waypoints in the 
TRACON airspace, and about one in ten crews 
did not comply with charted and cleared 
speeds at the final approach fix. (3) Pilots were 
very comfortable with the use of LNAV in the 
TRACON airspace, but at least one in four 
pilots expressed discomfort with or non- 
acceptance of the use of VNAV in the 
TRACON airspace for safety, performance, 
and/or situation awareness reasons. (4) One in 
five crews demonstrated that once taken off of 
the FMS procedure and then re-cleared for it, 
that they were reluctant to re-engage the 
VNAV mode. (5) Crews in general liked the 
FMS and CTAS/FMS procedures and found 
them acceptable. Crews mentioned the ability 
to look ahead and plan energy management as 
a primary positive contribution of the 
procedures. (6) Crews were comfortable with 
the use of datalink for route modifications in 
both the Center and TRACON airspace and 
expressed concern mainly for interfaces that 
produce excessive heads-down time or require 
manual entry of alpha-numeric information. 

(7) Crews demonstrated the best overall 
performance in terms of minimizing heads- 
down and dwell times using the DDL interface 
as compared to the CDU DL and MPE 
interfaces. 
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Figure 15. Pilot questionnaire responses to the question: How acceptable or unacceptable was the 
Glenn Rose FMS Arrival and Delmo FMS transition procedure as a whole? 
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Figure 16. Pilot questionnaire responses to the question: How acceptable or unacceptable was the 
requirement to use data link for route modifications while in the TRACON? 
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To validate the workload results gathered in 
this study, future simulation studies should 
include contexts that more closely match real- 
world operations. While creating this 
environment in a simulation setting is difficult, 
pilot recommendations gathered in the current 
study helped produce a list of practical steps 
for accomplishing this goal. Pilots reported 
that out-the-window (visual) traffic, a TCAS 
display, realistic radio chatter, adverse or 
unpredictable winds and weather, requirements 
for company communications, and cabin 
interruptions were the most conspicuous real- 
world elements missing from the current 
study. 

The drop in workload found in the baseline 
FMS procedure scenario runs raised the issue 
of possible under stimulation of pilots creating 
the potential for automation-induced 
complacency on the flight deck. As the air 
traffic system evolves, and FMS descent 
procedures become more commonplace, 
studies addressing this issue become necessary. 
While performance problems were more 
common in this study when crews were flying 
FMS procedures (e.g. poor speed control), this 
cannot be attributed directly to crew 
automation complacency as other factors 
previously mentioned likely played a role. An 
experimental design, well controlled and 
geared specifically toward the question of 
complacency is required. 


Because of the resistance by a few of the pilots 
to the use of VNAV in the TRACON, and the 
inherent speed control problems this mode 
introduces, the requirement to use VNAV in 
FMS procedures should be carefully 
considered. This is especially true for 
CTAS/FMS integration where speed control is 
vital to operational success. The question of 
what benefits are derived with the use of 
VNAV in the TRACON seems appropriate. 
Based on the results in this study, the VNAV 
requirement has been removed from the 
planned follow-up study on CTAS/FMS 
integration. 

Crew performance and acceptance of the 
TRACON route modifications required for 
CTAS/FMS integration is an encouraging 
finding. As with the FMS procedure workload 
findings, these results also need further study 
and validation in the more realistic simulation 
context outlined above. Also, the introduction 
of message errors and equipment malfunctions 
in future studies would provide a measure of 
crew error trapping capabilities with these 
interfaces. The fact that some crews failed to 
read the initial datalink message text and/or 
failed to review the loaded route raises 
concerns of crew complacency with this 
communication format. 
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Appendix A 

STAR, FMS, and CTAS/FMS Charts 
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Appendix B 

Participant Consent Form 




NASA Ames Research Center 
Human Research Minimal Risk Consent 

To the Test Subject: Please read this consent form and the attached protocol and/or subject 
instructions carefully. Make sure all questions have been answered to your satisfaction before 
signing. 

I agree to participate as a subject in CTAS/FMS Integration Crew Factors research experiment as 
described in the attached protocol or subject instructions. I understand that I am employed by 
Raytheon Corporation who can be contacted at (650) 604-2118. 

I understand that my participation could cause me minimal risk*, inconvenience, or discomfort. 

The purpose and procedures have been explained to me and I understand the risks and discomforts 
as described in the attached research protocol. 

To my knowledge, I have no medical conditions, including pregnancy, that will prevent my 
participation in this study. I understand that if my medical status should change while a participant in 
the research experiment that there may be unforeseeable risks to me (or the embryo or fetus if 
applicable). I agree to notify the Principal Investigator (P.I.) or medical monitor of any known 
changes in my condition for safety purposes 

My consent to participate as a subject has been freely given. I may withdraw my consent, and 
thereby withdraw from the study at any time without pearly or loss of benefits to which I am entitled. 

I understand that the P.I. may request my withdrawal or the study may be terminated for any reason. 

I agree to follow procedures for orderly and safe termination. 

I am not releasing NASA from liability for any injury arising as a result from my participation in 
this study. 

I hereby agree that all records collected by NASA in the course of this study are available to the 

research study investigators, support staff, and any duly authorized research review committee. I grant 
NASA permission to reproduce and publish all recorded, notes, or data collected from my 
participation, provided there is no association of my name with the collected data and that 
confidentiality is maintained, unless specifically waived by me. 

I have had the opportunity to ask questions and have received satisfactory answers to all my 

questions. I understand the P.I. for the study is the person responsible for this activity and that any 
pertinent questions regarding the research will be addressed to him/her during the course of the 
study. I have read the above agreement, the attached protocol and/or subject instructions prior to 
signature, and understand the contents. 

*Minimal risk means that the probability and magnitude of harm or discomfort anticipated in the 
research are not greater, in and of themselves, than those ordinarily encountered in daily life or 
during performance of routine physical or psychological examinations or tests. 
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Signature of Test Subject 

Date 

Signature of Principal Investigator 

Printed name of test subject 


Printed name of Principal Investigator 

Address 


Telephone Number of Principal Investigator 

City, State, Zip Code 


Subject Signature: Authorization for videotape 


Telephone Number of Test Subject Subject Signature: Authorization for release of 

information to Non-NASA Source 


For Questions regarding 
Serena Stanford at SJSU 
Everett Palmer at NASA 
Barry Crane at SJSU 


this study contact: 
(408) 924 2480 
(650) 604 0673 
(650) 604 201 1 
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Descent Scenarios 



Appendix C 
Descent Scenarios 


Baseline Current Day Procedure Scenario 


AIRCRAFT 

POSITION 

CLEARANCE PHRASEOLOGY 

CENTER: 127.15 

•IP: 10 miles 
southeast of Abilene 
on JEN3 Arrival 

(“NASA 21, cleared to DFW via the GLENN ROSE 3 Arrival, 
Abilene transition.”) 

• Following IP 

“NASA 21, descend via the GLENN ROSE 3 Arrival, cross 
FEVER at and maintain 11,000 feet.” 

• A/C at Glenn Rose 

“NASA 21. contact Regional Approach 133.62” 

FEEDER. 133.62 

• A/C at Glenn Rose 

“NASA 21, After FEVER, Maintain 11,000 feet, comply with 
speeds on STAR, expect vectors to ILS18R.” 

• A/C before 
DELMO 

“NASA 21, Contact Regional Approach 118.42.” 

FINAL: 118.42 

• A/C at 
DELMO 

“NASA 21, fly heading 350 vectors to ILS18R, descend and 
maintain 5000’.” 

• A/C between two 
miles before ROSEL 

“NASA 21, Reduce speed to 190, descend and maintain 3000.” 

• A/C -six miles 
beyond ROSEL 

“NASA 21, tum right heading 090.” 

• A/C on base 

“NASA 21, tum right heading 140. intercept ILS18R localizer. _ 

• A/C on final 

“NASA 21, cleared for ELS18R approach, cross Final Approach 
Fix at 170 knots.” 

• A/C on final 

“NASA 21. contact DFW tower 124.15” 

TOWER: 124.15 

• A/C on final 

“NASA 21, cleared to land runway 18R 


Baseline FMS Procedure Scenario 


AIRCRAFT 

POSITION 

PHRASEOLOGY 

CENTER: 127.15 

• IP: 10 miles 
southeast of Abilene 
on JEN FMS Arrival 

(“NASA 21, cleared to DFW via GLENN ROSE FMS Arrival, 
Abilene transition.”) 

• Following IP 

“NASA 21, descend via GLENN ROSE FMS 
Arrival” 

• A/C at Glenn Rose 

NASA 21. Contact Regional Approach 133.62 






FEEDER: 133.62 

• A/C at Glenn Rose 

“NASA 21, Roger, cleared for DELMO FMS Transition to 
ILS18R” 

• A/C before 
DELMO 

“NASA 21, Contact Regional Approach 118.42” 

FINAL: 118.42 

• A/C before 
DELMO 

“NASA 21 Roger” 

• A/C on base 

“NASA 21, cleared for ILS18R approach.” 

• A/C on final 

“NASA 21, Contact DFW tower 124.15” 

| TOWER: 124.15 

| • A/C on final 

“NASA 21, cleared to land runway 18R” 


Complex Current Day Procedure Scenario 


AIRCRAFT 

POSITION 

CLEARANCE PHRASEOLOGY 

| CENTER: 127.15 

•IP: 10 miles 
southeast of Abilene 
on JEN3 Arrival 

(“NASA 21, cleared to DFW via the GLENN ROSE 3 Arrival, 
Abilene transition.”) 

• Following DP 

“NASA 21, fly heading 080, descend pilots discretion, maintain 
FL240, expect to cross FEVER at 11,000 feet.” 

• A/C level at FL240 
for 25 secs. 

“NASA 21 traffic 2 o’ clock, five miles, northbound, 2000 
below you.” 

“NASA 21, traffic no factor, cleared direct FEVER, cross 
FEVER at and maintain 11,000 feet.” 

• A/C abeam Glenn 
Rose 

“NASA 21, contact Regional Approach 133.62.” 

I FEEDER. 133.62 

• A/C abeam Glenn 
Rose 

“NASA 21, After FEVER, Maintain 11,000 feet, comply with 
speeds on the STAR, expect vectors to runway 18R.” 

• A/C before 
DELMO 

“NASA 21, Contact Regional Approach 118.42” 

FINAL: 118.42 

• A/C at 
DELMO 

“NASA 21, fly heading 350 vectors to ILS18R, descend and 
maintain 5000’.” 

• A/C between six 
miles before ROSEL 

“NASA 21, Reduce speed to 190, descend and maintain 3000.” 

• A/C -six miles 
beyond ROSEL 

“NASA 21, turn right heading 090.” 

• A/C on base 

“NASA 21, turn right heading 140, intercept ILS18R localizer. 

• A/C on final 

“NASA 21, cleared for ILS18R approach, cross Final Approach 


48 






Fix at 170 knots.” 

• A/C on final 

“NASA 21. Contact DFW tower 124.15” 

TOWER: 124.15 

• A/C on final 

“NASA 21, Cleared to land runway 18R” 


Complex FMS Procedure Scenario 

AIRCRAFT 

POSITION 

PHRASEOLOGY 

1 CENTER: 127.15 

• DP: 10 miles 
southeast of Abilene 
on JEN FMS Arrival 

(“NASA 21, cleared to DFW via GLENN ROSE FMS Arrival, 
Abilene transition.”) 

• Following IP 

“NASA 21, fly heading 080, descend at pilots discretion, 
maintain FL240, expect direct FEVER to rejoin GLENN ROSE 
FMS Arrival” 

• A/C at FL240 for 
25 secs. 

“NASA 21 traffic 4 o’ clock, five miles, northwestbound, 2000 
below you.” 


“NASA 21, clear of traffic, cleared direct FEVER, resume 
GLENN ROSE FMS Arrival.” 

• A/C abeam Glenn 
Rose 

NASA 21, Contact Regional Approach 133.62 

FEEDER: 133.62 

• A/C abeam Glenn 
Rose 

“NASA 21, Roger, after crossing FEVER, cleared for DELMO 
FMS transition to ILS18R” 

• A/C before 
DELMO 

“NASA 21, Contact Regional Approach 118.42” 

FINAL: 118.42 

• A/C before 
DELMO 

“NASA 21 Roger” 

• A/C six miles prior 
to ROSEL 

“NASA 21, Slow to 190 knots.” 

• A/C just past 
ROSEL 

“NASA 21, Maintain current heading, standby for base turn 
clearance.” 

• A/C seven miles 
beyond ROSEL 

“NASA 21, tum right heading 090.” 

• A/C on base 

“NASA 21, tum right heading 140, intercept ILS18R localizer 

• A/C on final 

cleared for ILS18R approach, cross Final Approach Fix at 170 
knots” 

• A/C on final 

“NASA 21, Contact DFW tower 124.15” 

TOWER: 124.15 

• A/C on final 

“NASA 21, cleared to land runway 18R” 


Manual Path Extension Procedure Scenario 





AIRCRAFT 

POSITION 

PHRASEOLOGY 

I CENTER: 127.15 i 

• IP: 10 miles 
southeast of Abilene 
on JEN FMS Arrival 

(“NASA 21, cleared to DFW via the GLENN ROSE FMS 
Arrival, Abilene transition.”) 

* Following IP 

“NASA 21, fly heading 080, descend pilots discretion at 300 
knots, maintain FL240, expect direct FEVER to rejoin GLENN 
ROSE FMS Arrival” 

• A/C at FL240 for 
25 secs. 

“NASA 21 traffic 5 o’ clock, five miles, northeastbound, 2000 
below you.” 


“NASA 21, clear of traffic, resume descent at 300 knots, cleared 
direct FEVER, resume GLENN ROSE FMS Arrival.” 

• A/C abeam Glenn 
Rose 

NASA 21, Contact Regional Approach 133.62 

FEEDER: 133.62 

• A/C abeam Glenn 
Rose 

“NASA 21, Roger, cleared for DELMO FMS transition to 
ILS18R” 

• A/C before 
DELMO 

“NASA 21, Contact Regional Approach 118.42” 

FINAL: 118.42 

• A/C before 
DELMO 

“NASA 21, Roger” 

•A/C just past 
DELMO 

“NASA 21, make base turn at ROSEL plus x.x nautical miles.” 

• A/C six miles prior 
to ROSEL 

“NASA 21, Slow to 190 knots.” 

• A/C on base 

“NASA 21, cleared for ILS18R approach.” 

• A/C on final 

“NASA 21, Contact DFW tower 124.15” 

I TOWER: 124.15 

• A/C on final 

“NASA 21, cleared to land runway 18R” 
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CPU DL and DDL Procedure Scenarios 


AIRCRAFT 

POSITION 

PHRASEOLOGY 

CENTER: 127.15 

• IP: 10 miles 
southeast of Abilene 
on JEN FMS Arrival 

(“NASA 21, cleared to DFW via the GLENN ROSE FMS 
Arrival, Abilene transition.”) 

• Following IP 

“NASA 21, Expect route uplink.” 

• Following IP 

Data link message: NASA 21 

- ROUTE MODIFICATION 

REVISED FMS RTE TO FEVER 

• Following data link 
clearance 

“NASA 21, descend via revised FMS route to FEVER at 300 
knots.” 

•A/C at -FL260 

“NASA 21, maintain FL240 for traffic” 

• A/C at FL240 for 
25 secs. 

“NASA 21 traffic 3 o’ clock, five miles, northbound, 2000 
below you.” 


“NASA 21, clear of traffic, resume FMS descent at 300 knots.” 

• A/C abeam Glenn 
Rose 

NASA 21, Contact Regional Approach 133.62 

1 FEEDER: 133.62 

• A/C abeam Glenn 
Rose 

“NASA 21, Roger, cleared for DELMO FMS Transition to 
ILS18R” 

• A/C before 
DELMO 

“NASA 21, Contact Regional Approach 118.42” 

FINAL: 118.42 

• A/C before 
DELMO 

“NASA 21, Roger, expect route modification uplink.” 

•A/C just past 
DELMO 

Data link message: NASA 21 
-ROUTE MODIFICATION- 
REVISED FMS RTE TO LEGRE 

• A/C six miles prior 
to ROSEL 

“NASA 21, Slow to 190 knots.” 

• A/C on base 

“NASA 21, cleared for ILS18R approach.” 

• A/C on final 

“NASA 21, Contact DFW tower 124.15” 

1 TOWER: 124.15 

• A/C on final 

“NASA 21, cleared to land runway 18R” 




Appendix D 

Workload Dimension Descriptions and Workload Rating Sheet 



Appendix D 

Workload Dimension Descriptions and Workload Rating Sheet 


Workload Dimension 

Endpoints 

Description 

MENTAL DEMAND 

Low/High 

How much mental 
and perceptual 
activity was required 
(e.g. thinking, 
deciding, calculating, 
remembering, 
monitoring, looking, 
searching, etc)? Was 
the task easy or 
demanding, simple or 
complex, exacting or 
forgiving? 

PHYSICAL DEMAND 

Low/High 

How much physical 
activity was required 
(e.g. pushing, pulling, 
turning, controlling, 
activating, etc. )? Was 
the task easy 
demanding, slow or 
brisk, slack or 
strenuous, restful or 
laborious? 

TEMPORAL DEMAND 

Low/High 

How much time 
pressure did you feel 
due to the rate or 
pace at which the 
tasks or task elements 
occurred? Was the 
pace slow and 
leisurely or rapid and 
frantic? 

PERFORMANCE 

Good/Poor 

How successful do 
you think you were 
in accomplishing the 
goals of the task set 
by the experimenter 
(or yourself)? How 
satisfied were you 
with your 
performance in 
accomplishing these 





goals 

EFFORT 

Low/High 

How hard did you 
have to work 
(mentally and 
physically) to 
accomplish your level 
of performance? 

FRUSTRATION LEVEL 

Low/High 

How insecure, 
discouraged, irritated, 
stressed, and annoyed 
versus secure, 
gratified, content, j 

relaxed and 
complacent did you 
feel during the task? 
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Procedure Performance Criteria 


General criteria that apply to all procedures: 

Altitudes 

Comply with all charted and ATC specified altitude restrictions within plus or minus 300 feet 
as measured when passing over location (e.g. waypoint, navaid, etc.) of specified restriction. If 
compliance is not possible due to limitations in aircraft performance or other reason, inform ATC. 

Speeds 

Comply with all charted and ATC specified speed restriction within plus or minus 10 KIAS as 
measured when passing over location (e.g. waypoint, navaid, etc.) of specified restriction or when in 
phase of flight where a speed restriction has been specified (e.g. descent speed). If compliance is not 
possible due to limitations in aircraft performance or other reason, inform ATC. 

Criteria specific to FMS and CTAS/FMS procedures: 

Use of LNAV 

Unless given a heading vector by ATC, fly the entire FMS or CTAS/FMS procedure in the 
LNAV guidance mode until the point of Localizer capture . 

Use of VNAV 

When flying the DELMO FMS Transition, remain in the VNAV guidance mode until the 
point of Glideslope capture. 

Use of VNAV top of descent 

Unless given a heading vector by ATC, when flying the Glen Rose FMS Arrival, begin descent 
within plus or minus 5 miles of the VNAV calculated top of descent point. If compliance is not 
possible, inform ATC. 

Responding to, loading, and executing DELMO FMS Transition 

When cleared for the DELMO FMS Transition to ILS18R, correctly perform this activity. 
Correct performance of this activity is as follows: (1) Readback clearance to ATC, (2) access the 
DEP/ARR page, (3) select ILS18R on the runway menu, (4) select DELMO on the transition menu, 

(5) press Execute on the CDU, (6) select 2400 in the MCP altitude window. 

Responding to, loading and executing a Manual Path Extension clearance 

Upon receipt of a Manual Path Extension clearance, correctly perform this activity. Correct 
performance of this activity is as follows: the PNF must (1) acknowledge and readback the ATC 
cleared base tum distance, (2) line select the reference waypoint ROSEL into the CDU scratchpad, (3) 
enter a backslash (/) and the new base tum point value, (4) reselect the waypoint into the flightplan, 

(5) review the modification on the CDU and Navigation display, and (6) execute the modification by 
pressing the execute button on the CDU. 
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Responding to, loading and executing a CDU DL clearance 

Upon receipt of a CDU DL clearance, correctly perform this activity. Correct performance of 
this activity is as follows: the PNF must (1) press the ATC key on the CDU, (2) read the message text 
on the CDU page aloud to the other crew member, (3) receive confirmation from the other crew 
member, (4) load the new route by pressing a LOAD prompt, (5) press the LEGS key on the CDU, 

(6) review and verify the loaded message on the LEGS page and Navigation display, (7) execute the 
new route by pressing the execute button on the CDU, (8) press the ATC key on the CDU, and (9) 
accept the message by pressing the ACCEPT prompt. 

Responding to, loading and executing a DDL clearance 

Upon receipt of a DDL clearance, correctly perform this activity. Correct performance of this 
activity is as follows: the PNF must (1) read the message text on the E1CAS display aloud to the other 
crew member, (2) receive confirmation from the other crew member, (3) review and verify the loaded 
message on the LEGS page and Navigation display, (4) execute the new route by pressing the execute 
button on the CDU, and (5) accept the message by pressing the accept button on the glareshield. 
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Appendix F 
Crew Questionnaire 



CTAS/FMS Integration Evaluation: ACFS Crew Factors Study -- Fall 1998 

Pilot Questionnaire 
BACKGROUND INFORMATION 


Date: Crew Position: 

Approximate total hours: 


Approximate total hours flying “glass” aircraft: 


Please specify approximate total hours in each type aircraft 727 

737-200 DC -10 L10-11 

MD-80 737-300/500 747-400 

767 777 A300 A310_ 

A340 


DC-9 . 

747-200/300 _ 

_ 757 

A3 20 


Other type aircraft (please specify) 


Estimated number of times you have flown into DFW in the past 3 years 

Estimated number of times you have flown the Glen Rose 3 Arrival into DFW in the past 3 
years 

Instructions: Please circle the tick mark on each scale that best fits your response to 
the following questions or statements. Please also provide answers to the yes/no 
questions and comments or explanations where indicated. Please consider your 
responses carefully. Your responses will play an important role in the evaluation 
being conducted 

FMS ARRIVAL AND TRANSITION CHARTS 

How clear or unclear was the information presented on the Glenn Rose FMS Arrival and 
Delmo FMS Transition charts used in this study? ^ ^ 

Very Somewhat Borderline Somewhat Very 

Unclear Unclear Clear Clear 


How organized or unorganized was the information presented on the Glenn Rose FMS 
Arrival and Delmo FMS Transition charts used in this study? 

i i m i i 

v= Somewhat Borderline Somewhat Very 
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Unorganized Unorganized 


Organized Organized 


How adequate or inadequate were the Glenn Rose FMS Arrival and Delmo FMS Transition 
charts in helping you perform the procedures? 


Very Somewhat Borderline Somewhat Very 

Inadequate Inadequate Adequate Adequate 


Was there any information missing from the FMS Arrival and Transition charts that you 
would have liked to see? 

Yes No 

If yes, please explain 


Do you think the FMS Arrival and Transition charts should have contained less 
information, or less detailed information? 

Yes No 

If yes, please explain 


How acceptable or unacceptable were the Glenn Rose FMS Arrival and Delmo FMS 
Transition charts as a whole? 

L I j 1 

Very Somewhat Borderline Somewhat Very 

UnacceptableUnacceptable Acceptable Acceptable 
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FMS ARRIVAL AND TRANSITION CLEARANCE PHRASEOLOGY 


Initial FMS Arrival Descent Clearance 

“ NASA 21, Descend via the Glenn Rose FMS Arrival, maintain XXX Knots in the 
descent” 

Was any portion of this clearance phraseology unclear or confusing the first time you 
heard it? 

Yes No 

Please explain 


When you first received this clearance, was it clear to you what to do? 

Yes No 

Please explain 


Delmo FMS Transition Clearance 

“ NASA 21, Cleared for the Delmo FMS Transition to 1LS18R ” 

Was any portion of this clearance phraseology unclear or confusing the first time you 
heard it? 

Yes No 

Please explain 

When you first received this clearance, was it clear to you what to do? 

Yes No 

Please explain 


How acceptable or unacceptable was the Glenn Rose FMS Arrival and Delmo FMS 
Transition phraseology as a whole? ^ 

Very Somewhat Borderline Somewhat Very 

UnacceptableUnacceptable Acceptable Acceptable 
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PROCEDURE ACCEPTABILITY 


FMS Procedure : Questions in this section refer specifically to the Glenn Rose FMS 
Arrival and Delmo FMS Transition procedure without regard to the 
adjunct route modification procedures (i.e. Manual Path Extension, 
CDU DL, and DDR). 


Was any portion of the Glenn Rose FMS Arrival and Delmo FMS Transition procedure 
unclear or confusing? 

Yes No 

Please explain 


How acceptable or unacceptable was the requirement to set your MCP altitude window to 
the last crossing restriction on the routing of the Glenn Rose FMS Arrival and Delmo FMS 
Transition procedure? 

Very Somewhat Borderline Somewhat Very 

UnacceptableUnacceptable Acceptable Acceptable 


How acceptable or unacceptable was the requirement to use LNAV during the Arrival 
portion (at and above 1 1,000) of the Glenn Rose FMS Arrival and Delmo FMS Transition 
procedure? 

Very Somewhat Borderline Somewhat Very 

UnacceptableUnacceptable Acceptable Acceptable 

How comfortable or uncomfortable were you using LNAV during the Arrival portion of the 
Glenn Rose FMS Arrival and Delmo FMS Transition procedure? 

L I L, I I 

Very Somewhat Borderline Somewhat Very 

UncomfortableUncomfortable ComfortableComfortable 


How acceptable or unacceptable was the requirement to use VNAV during the Arrival 
portion of the Glenn Rose FMS Arrival and Delmo FMS Transition procedure? 
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Very Somewhat 

UnacceptableUnacceptable 


Borderline 


Somewhat Very 

Acceptable Acceptable 


How comfortable or uncomfortable were you using VNAV during the Arrival portion of the 
Glenn Rose FMS Arrival and Delmo FMS Transition procedure? 

I | _]_ I , l 

Somewhat Borderline Somewhat Very 

U ncomfortableU ncomfortable ComfortableComfortable 


How acceptable or unacceptable was the requirement to use LNAV during the Delmo 
Transition portion (below 1 1 ,000) of the Glenn Rose FMS Arrival and Delmo FMS 
Transition procedure? 

1 | L I 

Very Somewhat Borderline Somewhat Very 

UnacceptableUnacceptable Acceptable Acceptable 


How comfortable or uncomfortable were you using LNAV during the Delmo Transition 
portion of the Glenn Rose FMS Arrival and Delmo FMS Transition procedure? 

L | Il_ I I 

Very Somewhat Borderline Somewhat Very 

UncomfortableUncomfortable ComfortableComfortable 


How acceptable or unacceptable was the requirement to use VNAV during the Delmo 
Transition portion of the Glenn Rose FMS Arrival and Delmo FMS Transition procedure? 

i i X i „ 1 

Very Somewhat Borderline Somewhat Very 

UnacceptableUnacceptable Acceptable Acceptable 


How comfortable or uncomfortable were you using VNAV during the Delmo Transition 
portion of the Glenn Rose FMS Arrival and Delmo FMS Transition procedure? 

L | _L_ I 1 

Very Somewhat Borderline Somewhat Very 

U ncomfortableU ncomfortable ComfortableComf ortab le 


“Nasa 21, fly heading 080, descend at pilot’s discretion, maintain FL240, expect direct 
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FEVER to rejoin Glenn Rose F MS Arrival”.... 

“Nasa 21, clear of traffic, cleared direct FEVER, resume Glenn Rose FMS Arrival.” 
How acceptable or unacceptable were the above set of clearances that vectored you off the 
Glenn Rose FMS Arrival and then had you resume it? 

Very Somewhat Borderline Somewhat Very 

UnacceptableUnacceptable Acceptable Acceptable 


“Nasa 21, Maintain current heading, standby for base turn clearance.” 

How acceptable or unacceptable was the above clearance given during the Delmo FMS 
Transition procedure? 

I I I 1 

Very Somewhat Borderline Somewhat Very 

UnacceptableUnacceptable Acceptable Acceptable 


Did you feel rushed at any time during the Glenn Rose FMS Arrival and Delmo FMS 
Transition procedure ? 

Yes No 

Please explain 


In general, how was your workload affected when flying the Glenn Rose FMS Arrival and 
Delmo FMS Transition procedure as compared to your current day descent procedures: 

I I _L_ l i 

Greatly Somewhat Unaffected Somewhat Greatly 

Decreased Decreased Increased Increased 


In general, how was your monitoring behavior affected when flying the Glenn Rose FMS 
Arrival and Delmo FMS Transition procedure as compared to your current day descent 
procedures: 

Greatly Somewhat Unaffected Somewhat Greatly 

Decreased Decreased Increased Increased 


The Glenn Rose FMS Arrival and Delmo FMS Transition procedure made flying the 
descent: 

I I L I 

Very Somewhat Borderline Somewhat Very 

Easy Easy Difficult Difficult 
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Compared to current day descent procedures, the Glenn Rose FMS Arrival and Delmo 
FMS Transition procedure was: 


Very Somewhat Borderline Somewhat Very 

Undesirable Undesirable Desirable Desirable 


How acceptable or unacceptable was the Glenn Rose FMS Arrival and Delmo FMS 
Transition procedure as a whole? ^ 

Very Somewhat Borderline Somewhat Very 

UnacceptableUnacceptable Acceptable Acceptable 


Please provide any comments you have regarding the Glenn Rose FMS Arrival and Delmo 
FMS Transition Procedure : 


CTAS/FMS PROCEDURES 


General 

How acceptable or unacceptable was the requirement to use data link for route modifications 
while in cruise near the top-of-descent (i.e. " Revised FMS route to FEVER”)? 

I 1 _L I 1 

Very Somewhat Borderline Somewhat Very 

UnacceptableUnacceptable Acceptable Acceptable 
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. . ..while in the TRACON? 

I „_| I I 

Very Somewhat Borderline Somewhat Very 

UnacceptableUnacceptable Acceptable Acceptable 


How acceptable or unacceptable was the issuance of a descent speed in the Glenn Rose 
FMS Arrival? 

I I I | 

Very Somewhat Borderline Somewhat Very 

UnacceptableUnacceptable Acceptable Acceptable 


In general, how acceptable or unacceptable was the requirement to make FMS edits to the 
Delmo FMS transition routing while on downwind in the TRACON airspace 9 

I I I i 1 

Very Somewhat Borderline Somewhat Very 

UnacceptableUnacceptable Acceptable Acceptable 


General Data link 

How acceptable or unacceptable was the data link phraseology used in the Center airspace 
(i.e. “ Revised FMS route to FEVER” )? 

L I I, I 1 

Very Somewhat Borderline Somewhat Very 

UnacceptableUnacceptable Acceptable Acceptable 


How acceptable or unacceptable was the data link phraseology used in the TRACON 
airspace (i.e. “ Revised FMS route to LEGRE” )? 

I I I I 

Very Somewhat Borderline Somewhat Very 

UnacceptableUnacceptable Acceptable Acceptable 


Manual Path Extension Procedure 

How acceptable or unacceptable was the Manual Path Extension procedure requirement to 
manually enter the modified base turn point into the CDU during the Delmo FMS 
Transition procedure? 

I I I I 1 

Very Somewhat Borderline Somewhat Very 

UnacceptableUnacceptable Acceptable Acceptable 


How acceptable or unacceptable were the tasks required to complete the Manual Pafh 
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Extension procedure ? ^ 

Very Somewhat Borderline Somewhat Very 

UnacceptableUnacceptable Acceptable Acceptable 


Did you feel rushed at any time during the Manual Path Extension procedure? 

Yes No 

Please explain 


How acceptable or unacceptable was the Manual Path Ext ension procedure phraseolo g y 
( “Make base turn at Rosel +X.X miles ”)? ^ 

Very Somewhat Borderline Somewhat Very 

UnacceptableUnacceptable Acceptable Acceptable 


How acceptable or unacceptable was the Manual Path Extension procedure as a whole? 

| | _L I „ 

Very Somewhat Borderline Somewhat Very 

UnacceptableUnacceptable Acceptable Acceptable 


Please provide any comments you have regarding the Manual Path Extension procedure : 


CPU DL Procedures 

How acceptable or unacceptable were the tasks required to complete the CDUDL 
procedure ? ^ ^ 

Very Somewhat Borderline Somewhat Very 


71 





U nacceptableUnacceptable 


Acceptable Acceptable 


How acceptable or unacceptable was the CPU DL interface for route modifications received 
in the Center airspace (“ Revised route to EFVER ”)? 

I . I L I l 

Very Somewhat Borderline Somewhat Very 

UnacceptableUnacceptable Acceptable Acceptable 


Did you have any trouble understanding the CPU DL data link message you received in the 
Center airspace once it was loaded 
(i.e. “ Revised FMS route to FEVER ” )? 

Yes No 

Please explain 


How acceptable or unacceptable was the CPU DL interface for route modifications received 
in the TRACON airspace ( “ Revised route to LEGRE”)1 

I I I I 1 

Very Somewhat Borderline Somewhat Very 

UnacceptableUnacceptable Acceptable Acceptable 


Did you have any trouble Understanding the CDU DL data link message you received in 
the TRACON airspace once it was loaded 
(i.e. "Revised FMS route to LEGRE” )? 

Yes No 

If yes, please explain 


Did you feel rushed at any time when completing the CDU DL procedure in the Center 
airspace? 

Yes No 
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If yes, please explain 


Did you feel rushed at any time when completing the CPU DL procedure in the TRACON 
airspace? 

Yes No 

If yes, please explain 


How acceptable or unacceptable was the CPU DL In terface as a whole? 

1 | _]_ I 

Very Somewhat Borderline Somewhat Very 

UnacceptableUnacceptable Acceptable Acceptable 


How acceptable or unacceptable was the CPU DL Procedure as a whole? 

Very Somewhat Borderline Somewhat Very 

UnacceptableUnacceptable Acceptable Acceptable 


Please provide any comments you have regarding the CPU DL procedure - 


DDR Procedures 

How acceptable or unacceptable were the tasks required to complete the DDR procedure? 

I | L_ I — 1 

Very Somewhat Borderline Somewhat Very 

UnacceptableUnacceptable Acceptable Acceptable 


How acceptable or unacceptable was the DDR interface for route modifications received in 
the Center airspace (“Revised route to FEVER ”)? 

i i _I i l 

Very Somewhat Borderline Somewhat Very 

UnacceptableUnacceptable Acceptable Acceptable 
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Did you have any trouble understanding the DDR data link message you received in the 
Center airspace (i.e. ‘‘Revised FMS route to FEVER ” )? 

Yes No 

Please explain 


How acceptable or unacceptable was the DDR interface for route modifications received in 
the TRACON airspace (“ Revised route to LEGRE ")? 

Very Somewhat Borderline Somewhat Very 

UnacceptableUnacceptable Acceptable Acceptable 


Did you have any trouble deciphering the DDR data link message you received in the 
TRACON airspace (i.e. ‘‘Revised FMS route to LEGRE ” )? 

Yes No 

Please explain 


Did you feel rushed at any time when completing the DDR procedure in the Center 
airspace? 

Yes No 

Please explain 

Did you feel rushed at any time when completing the DDR procedure in the TRACON 
airspace? 

Yes No 

Please explain 


How acceptable or unacceptable was the DDR Interface as a whole? 

I I I | 

Very Somewhat Borderline Somewhat Very' 

UnacceptableUnacceptable Acceptable Acceptable 
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How acceptable or unacceptable was the DDR Procedure as a whole? 


Very Somewhat Borderline 

UnacceptableUnacceptable 


Somewhat Very 

Acceptable Acceptable 


Please provide any comments you have regarding the DDR procedure: 


Please state the FMS route modification interface (e.g. MPE, DDR, CDU DL) that you 
most preferred. 


Why? 


Which interface(s) do you foresee as most likely to be used in day to day operations? 


AUTOMATION 

Indicate the degree to which you feel you were able to anticipate automation behavior while 
flying the descents , . 

I I I I l 

Never Not Often Sometimes Usually Always 

Were you ever confused by the automation behavior during any of the descents flown? 

Yes No 

If yes, please explain 

Were you ever surprised by the automation behavior during any of the descents flow n? 

Yes No 

If yes, please explain 
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Did you feel “ahead” of the aircraft when flying the FMS and CTAS/FMS procedures 7 

I 1 I i i 

Never Not Often Sometimes Usually Always 


Did the increased use of high level automation with the FMS and CTAS/FMS procedures 
impact your crew coordination activities in any way? 

Yes No 

If yes, please explain the impact they had? 


Did the increased use of high level automation with the FMS and CTAS/FMS procedures 
allow you more or less time to look out the window? 

Much Somewhat Borderline Somewhat Much 

Less Time Less Time More Time More Time 


With the FMS and CTAS/FMS procedures, how adequately or inadequately do you feel 
you were able to picture the vertical situation of the descent given the current displays 7 

i i Zl i i 

Very Somewhat Borderline Somewhat Very 

Inadequatelylnadequately Adequately Adequately 


ADEQUACY OF BRIEFING AND TRAINING 


Did the Glenn Rose FMS Arrival and Delmo FMS Transition briefing you received prepare 
you sufficiently for this procedure? 

Yes No 

(please elaborate) 


Did the Manual Path Extension briefing and training you received prepare you sufficiently 
for this procedure? 

Yes No 
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(please elaborate) 


Did the CPU DL briefing and training you received prepare you sufficiently for this 
procedure? 

Yes No 

(please elaborate) 


Did the DDR briefing and training you received prepare you sufficiently for this procedure? 

Yes No 

(please elaborate) 


Do you think simulator training is needed for introduction of the Glenn Rose FMS Arrival 
and Delmo FMS Transition Procedure? 

Yes No 

(if so, why?) 


OTHER 

How many descents did it take for the Glenn Rose FMS Arrival and Delmo FMS 
Transition to become comfortable, or routine? 


Describe any techniques you may have developed for flying the Glenn Rose FMS Arrival 
and Delmo FMS Transition. 
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Appendix G 

Crew Questionnaire Responses 
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Pilot Questionnaire Responses to Likert scale and ves or n o questions 
FMS ARRIVAL AND TRANSITION CHARTS 

How clear or unclear was the information presented on the Glenn Rose FMS Arrival and 


Delmo FMS Transition charts used in this study? 



0 

1 

0 

1 

0 

1 . 

7 

1 

li 

1 

Very 

Unclear 

Somewhat 

Unclear 

Borderline 

Somewhat 

Clear 

Very 

Clear 


How organized or unorganized was the information presented on the Glenn Rose FMS 
Arrival and Delmo FMS Transition charts used in this study? 

0 0 0 9 14 

1 i L i i 

Very Somewhat Borderline Somewhat Very 

Unorganized Unorganized Organized Organized 


How adequate or inadequate were the Glenn Rose FMS Arrival and Delmo FMS 
Transition charts in helping you perform the procedures? 

0 0 1 6 17 


Very Somewhat Borderline Somewhat Very 

Inadequate Inadequate Adequate Adequate 


Was there any information missing from the FMS Arrival and Transition charts that you 
would have liked to see? 

Yes 2 No 21 

Do you think the FMS Arrival and Transition charts should have contained less 
information, or less detailed information? 

Yes 6 No 18 
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How acceptable or unacceptable were the Glenn Rose FMS Arrival and Delmo FMS 
Transition charts as a whole? 


0 0 
1 1 

1 

1 

8 

1 

] 

Very Somewhat 

U nacceptableUnacceptable 

Borderline 

Somewhat 

Acceptable 

Very 

Acceptable 


FMS ARRIVAL AND TRANSITION CLEARANCE PHRASEOLOGY 
hutial FMS. Arrival Descent Clearance 

“NASA 21, Descend via the Glenn Rose FMS Arrival, maintain XXX Knots in the 
descent” 

Was any portion of this clearance phraseology unclear or confusing the first time you 
heard it? 

Yes 5. No 19 

When you first received this clearance, was it clear to you what to do? 

Yes 20 No 4 

Delmo FMS Transition Cleararice 

“NASA 21, Cleared for the Delmo FMS Transition to ILS18R” 

Was any portion of this clearance phraseology unclear or confusing the first time you 
heard it? 

Yes 4 No 20 

When you first received this clearance, was it clear to you what to do? 

Yes 21 No 4 

How acceptable or unacceptable was the Glenn Rose FMS Arrival and Delmo FMS 
Transition phraseology as a whole? 


0 2 

1 1 

0 

J 

7 

1 

15 

1 

Very Somewhat 

UnacceptableUnacceptable 

Borderline 

Somewhat 

Acceptable 

Very 

Acceptable 
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PROCEDURE ACCEPTABILITY 

FMS Procedure : Questions in this section refer specifically to the Glenn Rose FMS 
Arrival and Delmo FMS Transition procedure without regard to the 
adjunct route modification procedures (i.e. Manual Path Extension, 
CDU DL, and DDR). 


Was any portion of the Glenn Rose FMS Arrival and Delmo FMS Transition procedure 
unclear or confusing? 

Yes 4 No 20 

How acceptable or unacceptable was the requirement to set your MCP altitude window to 
the last crossing restriction on the routing of the Glenn Rose FMS Arrival and Delmo FMS 
Transition procedure? 


1 1 

1 1 

4 

1 

10 

1 

8 

1 

Very Somewhat 

UnacceptableUnacceptable 

Borderline 

Somewhat 

Acceptable 

Very 

Acceptable 


How acceptable or unacceptable was the requirement to use LNA V during the Arriv al 
portion (at and above 1 1,000) of the Glenn Rose FMS Arrival and Delmo FMS Transition 
procedure? 


Viry Somewhat Borderline Somewhat Very 

UnacceptableUnacceptable Acceptable Acceptable 

How comfortable or uncomfortable were you using LNAV during the Arrival portion of the 
Glenn Rose FMS Arrival and Delmo FMS Transition procedure? 


0 0 0 6 

Very Somewhat Borderline Somewhat Very 

UncomfortableUncomfortable ComfortableComfortable 


How acceptable or unacceptable was the requirement to use VNAV during the Arrival 
portion of the Glenn Rose FMS Arrival and Delmo FMS Transition procedure? 


Very Somewhat Borderline Somewhat Very 
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UnacceptableUnacceptable 


Acceptable Acceptable 


How comfortable or uncomfortable were you using VNAV during the Arrival portion of the 
Glenn Rose FMS Arrival and Delmo FMS Transition procedure? 

211 8 12 

Very Somewhat Borderline Somewhat Very 

U ncomfor tableU ncomfortable ComfortableComfortable 


How acceptable or unacceptable was the requirement to use LNAV during the Delmo 
Transition portion (below 1 1,000) of the Glenn Rose FMS Arrival and Delmo FMS 


Transition procedure? 




0 0 

1 l 

1 

6 

1 

17 

1 

Very Somewhat 

U nacceptableU nacceptable 

Borderline 

Somewhat 

Acceptable 

Very 

Acceptable 


How comfortable or uncomfortable were you using LNAV during the Delmo Transition 
portion of the Glenn Rose FMS Arrival and Delmo FMS Transition procedure? 

0 0 0 4 19 

1 I 1 I I 

Very Somewhat Borderline Somewhat Very 

UncomfortableUncomfortable ComfortableComfortable 


How acceptable or unacceptable was the requirement to use VNAV during the Delmo 
Transition portion of the Glenn Rose FMS Arrival and Delmo FMS Transition procedure? 


1 3 

1 „ l 

3 

... 1 

6 

1 

11 

1 

Very Somewhat 

UnacceptableUnacceptable 

Borderline 

Somewhat 

Acceptable 

Very 

Acceptable 


How comfortable or uncomfortable were you using VNAV during the Delmo Transition 
portion of the Glenn Rose FMS Arrival and Delmo FMS Transition procedure? 

2 3 2 6 11 

I I L I 1 

Very Somewhat Borderline Somewhat Very 
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UncomfortableUncomfortable 


ComfortableComfortable 


“Nasa 21, fly heading 080, descend at pilot's discretion, maintain FL240, expect direct 
FEVER to rejoin Glenn Rose FMS Arrival’'.... 

“Nasa 21, clear of traffic, cleared direct FEVER, resume Glenn Rose FMS Arrival 
How acceptable or unacceptable were the above set of clearances that vectored you off the 
Glenn Rose FMS Arrival and then had you resume it? 

01 1 11 10 

i I L i __[ 

Very Somewhat Borderline Somewhat Very 

UnacceptableUnacceptable Acceptable Acceptable 


“Nasa 21, Maintain current heading, standby for base turn clearance.” 

How acceptable or unacceptable was the above clearance given during the Delmo FMS 


Transition procedure? 




0 0 

1 1 

i 

l 

6 

1 

16 

1 

Very Somewhat 

UnacceptableUnacceptable 

Borderline 

Somewhat 

Acceptable 

Very 

Acceptable 


Did you feel rushed at any time during the Glenn Rose FMS Arrival and Delmo FMS 
Transition procedure ? 

Yes 2 No 21 


In general, how was your workload affected when flying the Glenn Rose FMS Arrival and 
Delmo FMS Transition procedure as compared to your current day descent procedures: 


4 

1 

9 

1 

5 

1 

5 

1 

0 

1 

Greatly 

Decreased 

Somewhat 

Decreased 

Unaffected 

Somewhat 

Increased 

Greatly 

Increased 


In general, how was your monitoring behavior affected when flying the Glenn Rose FMS 
Arrival and Delmo FMS Transition procedure as compared to your current day descent 
procedures: 
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Greatly 
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Unaffected 

Somewhat 

Increased 

Greatly 
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The Glenn Rose FMS Arrival and Delmo FMS Transition procedure made flvinp the 
descent: 

8 

1 

11 

l 

3 

__ 1 

2 

1 

0 

1 

Very 

Easy 

Somewhat 

Easy 

Borderline 

Somewhat 

Difficult 

Very 

Difficult 


Compared to current day descent procedures, the Glenn Rose FMS Arrival and Delmo 
FMS Transition procedure was: 


0 2 

L 1 

1 

1 

10 

1 

11 

1 

Very Somewhat 

Undesirable Undesirable 

Borderline 

Somewhat 

Desirable 

Very 

Desirable 


How acceptable or unacceptable was the Glenn Rose FMS Arrival and Delmo FMS 
Transition procedure as a whole? 


0 1 
1 1 

2 

.... 1 

11 

1 

10 

Very Somewhat 

UnacceptableUnacceptable 

Borderline 

Somewhat 

Acceptable 

Very 

Acceptable 


CTAS/FMS PROCEDURES 


General 


How acceptable or unacceptable was the requirement to use data link for route modifications 
while in cruise near the top-of-descent (i.e. "Revised FMS route to FEVER ”)? 
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1 

0 

1 
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Very 

Somewhat 

Borderline 

Somewhat 

Very 

U nacceptableUnacceptable 


Acceptable 

Acceptable 

. . . .while in the TRACON? 
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Very 
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Borderline 
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Very 

UnacceptableUnacceptable 


Acceptable 

Acceptable 
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How acceptable or unacceptable was the issuance of a descent speed in the Glenn Rose 
FMS Arrival? 


0 0 

1 1 

i 

l 

10 

L 

13 

1 

Very Somewhat 

UnacceptableUnacceptable 

Borderline 

Somewhat 

Acceptable 

Very 

Acceptable 


In general, how acceptable or unacceptable was the requirement to make FMS edits to the 
Delmo FMS transition routing while on downwind in the TRACON airspace? 


ViF y Somewhat Borderline Somewhat ~ Very 

UnacceptableUnacceptable Acceptable Acceptable 


General Data link 


How acceptable or unacceptable was the data link phraseology used in the Center airspace 
(i.e. “ Revised FMS route to FEVER ” )? 


0 0 1 

| I I 

Very Somewhat Borderline 

UnacceptableUnacceptable 


4 19 

Somewhat Very 

Acceptable Acceptable 


How acceptable or unacceptable was the data link phraseology used in the TRACON 
airspace (i.e. “Revised FMS route to LEGRE” )? 


0 0 0 

I | I 

Very Somewhat Borderline 

UnacceptableUnacceptable 


3 21 

l 1 

Somewhat Very 

Acceptable Acceptable 
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Manual Path Extension Procedure 


How acceptable or unacceptable was the Manual Path Extension procedure requirement to 
manually enter the modified base turn point into the CDU during the Delmo FMS 
Transition procedure? 

0 13 12 8 

Very Somewhat Borderline Somewhat Very 

UnacceptableUnacceptable Acceptable Acceptable 


How acceptable or unacceptable were the tasks required to complete the Manual Path 
Extension procedure ? 


0 1 
1 v [ 
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1 

10 
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11 

1 

Very Somewhat 

UnacceptableUnacceptable 

Borderline 

Somewhat 

Acceptable 

Very 

Acceptable 


Did you feel rushed at any time during the Manual Path Extension procedure ? 
Yes J No 22 


How acceptable or unacceptable was the Manual Path Extension procedure phraseology 
("Make base turn at Rose! +X.X miles”)? 


0 0 
1 „ I 
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Very Somewhat 

UnacceptableUnacceptable 

Borderline 

Somewhat 

Acceptable 

Very 

Acceptable 


How acceptable or unacceptable was the Manual Path Extension procedure as a whole? 


0 1 

L 1 
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13 
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Very Somewhat 

UnacceptableUnacceptable 

Borderline 

Somewhat 

Acceptable 
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Acceptable 


CDU DL Procedures 

How acceptable or unacceptable were the tasks required to complete the CDU DL 


88 







procedure ? 


0 0 
1 1 

0 

1 

8 

1 

15 

1 

Very Somewhat 

UnacceptableUnacceptable 

Borderline 

Somewhat 

Acceptable 

Very 

Acceptable 


How acceptable or unacceptable was the CPU DL interface for route modifications received 
in the Center airspace ( “ Revised route to EFVER ”)? 

0 0 0 8 15 

1 i L i „ — i 

Very Somewhat Borderline Somewhat Very 

UnacceptableUnacceptable Acceptable Acceptable 


Did you have any trouble understanding the CPU DL data link message you received in the 
Center airspace once it was loaded 
(i.e. “Revised FMS route to FEVER” )? 

YesO No 24 


How acceptable or unacceptable was the CPU DL interface for route modifications received 
in the TRACON airspace (“ Revised route to LEGRE ")? 


0 1 4 

1 1 L_ 

Very Somewhat Borderline 

UnacceptableUnacceptable 


8 11 

i v I 

Somewhat Very 

Acceptable Acceptable 


Did you have any trouble Understanding the CDU DL data link message you received in 
the TRACON airspace once it was loaded 
(i.e. “Revised FMS route to LEGRE” )? 

Yes 0 No 24 


Did you feel rushed at any time when completing the CDU DL procedure in the Center 
airspace? 

YesO No 24 

Did you feel rushed at any time when completing the CDU DL procedure in the TRACON 
airspace? 

Yes 2 No 22 
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How acceptable or unacceptable was the CPU DL Interface as a whole? 


0 0 

L 1 
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.. 1 

12 

1 
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Very Somewhat 

UnacceptableUnacceptable 

Borderline 

Somewhat 

Acceptable 

Very 

Acceptable 


How acceptable or unacceptable was the CPU DL Procedure as a whole? 
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l 
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11 
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Somewhat 

Borderline 

Somewhat 

Very 

UnacceptableUnacceptable 


Acceptable 

Acceptable 

DDR Procedures 




How acceptable or unacceptable 

: were the tasks required to complete the 

DDR procedure? 
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, 1 
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1 

17 

1 

Very 

Somewhat 

Borderline 

Somewhat 

Very 

UnacceptableUnacceptable 


Acceptable 

Acceptable 


How acceptable or unacceptable was the DDR interface for route modifications received in 
the Center airspace ( “Revised route to FEVER ”)? 


0 0 

L 1 
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J 

6 

1 

16 

1 

Very Somewhat 

UnacceptableUnacceptable 

Borderline 

Somewhat 

Acceptable 

Very 

Acceptable 


Did you have any trouble understanding the DDR data link message you received in the 
Center airspace (i.e. “Revised FMS route to FEVER ” )? 

YesO No 24 
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How acceptable or unacceptable was the DDR interface for route modifications received in 
the TRACON airspace ( ‘‘Revised route to LEGRE ”)? 


0 0 

1 1 

3 

1 

6 

1 


Very Somewhat 

UnacceptableUnacceptable 

Borderline 

Somewhat 

Acceptable 

Very 

Acceptable 


Did you have any trouble deciphering the DDR data link message you received in the 
TRACON airspace (i.e. “Revised FMS route to LEGRE" )? 

YesO No 24 


Did you feel rushed at any time when completing the DDR procedure in the Cente r 
airspace? 

YesO No 24 

Did you feel rushed at any time when completing the DDR procedure in the TRACON 
airspace? 

Yes 2 No 22 


How acceptable or unacceptable was the DDR Interface as a whole? 


0 0 

I I 

Very Somewhat 

U nacceptableU nacceptable 


3 7 14 

I I l 

Borderline Somewhat Very 

Acceptable Acceptable 


How acceptable or unacceptable was the DDR Procedure as a whole? 


0 0 1 

1 [ I 

Very Somewhat Borderline 

U nacceptableU nacceptable 


8 15 

I 1 

Somewhat Very 

Acceptable Acceptable 
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AUTOMATION 


Indicate the degree to which you feel you were able to anticipate automation behavior while 


flying the descents 




0 0 

L 1 

3 

1 

18 

1 

1 

1 

Never Not Often 

Sometimes 

Usually 

Always 


Were you ever confused by the automation behavior during any of the descents flown? 
YesJJ No 9 


Were you ever surprised by the automation behavior during any of the descents flown? 
Yes 12 No 12 


Did you feel “ahead” of the aircraft when flying the FMS and CTAS/FMS procedures? 


0 

1 
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1 

3 

. 1 

12 

1 

9 

1 

Never 

Not Often 

Sometimes 

Usually 

Always 


Did the increased use of high level automation with the FMS and CTAS/FMS procedures 
impact your crew coordination activities in any way? 

Yes 8 No 16 


Did the increased use of high level automation with the FMS and CTAS/FMS procedures 
allow you more or less time to look out the window? 
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Much 

Somewhat 

Borderline 

Somewhat 

Much 

Less Time 

Less Time 


More Time 

More Time 
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With the FMS and CTAS/FMS procedures, how adequately or inadequately do you feel 
you were able to picture the vertical situation of the descent given the current displays . 


Very Somewhat Borderline Somewhat " Very 

Inadequatelylnadequately Adequately Adequately 


ADEQUACY OF BRIEFING AND TRAINING 


Did the Glenn Rose FMS Arrival and Delmo FMS Transition briefing you received prepare 
you sufficiently for this procedure? 

Yes 24 No 0 


Did the Manual Path Extension briefing and training you received prepare you sufficiently 
for this procedure? 

Yes 24 NoO 


Did the CPI I DL briefing and training you received prepare you sufficiently for this 
procedure? 

Yes 24 NoO 

Did the DPR hnefinp and training you received prepare you sufficiently for this procedure? 
Yes 24 NoO 

Do you think simulator training is needed for introduction of the Glenn Rose FMS Arrival 
and Delmo FMS Transition Procedure? 

Yes 9 No 15 

OTHER 

Did you ever feel bored when flying any of the FMS or CTAS/FMS descents? 

Yes JA No 10 

If you were to fly these procedures regularly, do you think boredom would become an 
issue? 

Yes 5 No 14 
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